{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 第二章 机器学习分类算法\n",
    "\n",
    "本章，我们要学习第一个公认的机器学习分类算法：感知机(perceptron)和自适应线性神经元(adaptive linear neurons)。\n",
    "\n",
    "首先我们用Python实现感知机算法，然后将其应用于Iris数据集。编码过程可以帮助我们更好地理解分类的概念以及如何用Python实现算法。\n",
    "\n",
    "学习自适应线性神经元算法会涉及到优化(optimization)相关的知识，为我们第三章运用scikit-learn中更加复杂的分类器打好数学基础。\n",
    "\n",
    "\n",
    "本章涉及到的知识点包括：\n",
    "* 对机器学习算法有直观了解\n",
    "* 使用pandas、NumPy和matplotlib读取数据、处理数据和数据可视化\n",
    "* 使用Python实现简单的线性分类算法\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 透过人工神经元一窥机器学习早期历史\n",
    "\n",
    "在我们讨论感知机及其相关算法细节前，先让我们回顾一下机器学习早期的发展历程。为了了解大脑工作原理进而设计通用人工智能算法，Warren McCullock和Walter Pitts 在1943年首次提出了简化版脑细胞的概念，即McCullock-Pitts(MCP)神经元(*A Logical Calculus of the Ideas\n",
    "Immanent in Nervous Activity*)。神经元是大脑的神经细胞，作用是处理、传播电信号，可见下图：\n",
    "![](https://ooo.0o0.ooo/2016/06/10/575a6855e1200.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "MCP神经元：有多个输入传递到树突，然后在神经元内部进行输入整合，如果累积的信号量超过某个阈值，会产生一个输出信号并且通过轴突进行传递。可以把这种神经细胞看作带有两个输出的简单逻辑门。\n",
    "\n",
    "十几年后，基于MCP神经元模型，Frank Rosenblatt发表了第一个感知机学习规则(*The Perceptron, a Perceiving and Recognizing Automaton.* Cornell Aeronautical Laboratory, 1957)。\n",
    "基于此感知机规则，Rosenblatt提出了能够自动学习最优权重参数的算法。啥是权重参数呢？权重就是输入特征的系数。\n",
    "除了能够学习权重参数，这个算法还能够预测一个样本是属于类别A还是类别B。\n",
    "\n",
    "我们将上面提到的“样本属于哪一个类别”这个问题称为二分类问题(binary classification task)，把其中涉及到的两个类别记作1(表示正类)和-1(表示负类)。我们再定义一个称为激活函数(activation function) 的东东$\\phi(z)$，其中$z$被称为激活函数输入($z=w_{1}x_{1}+...+w_{m}x_{m}$)。\n",
    "\n",
    "$x$是输入向量, $x=[x_{1}, ..., x_{m}]^{T}$，$w$是相应的权重参数向量,$w=[w_{1}, ..., w_{m}]^{T}$。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "此时，如果某个样本$x^{(i)}$的激活值，即$\\phi(z)$大于事先设置的阈值$\\theta$,我们就说样本$x^{(i)}$属于类别1，否则属于类别-1。\n",
    "\n",
    "在感知机学习算法中，激活函数$\\phi(\\cdot)$的形式非常简单，仅仅是一个单位阶跃函数(也被称为Heaviside阶跃函数):\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/10/575a6f866716d.png)\n",
    "为了方便推导，我们可以将阈值$\\theta$挪到不等式左边并且额外定义一个权重参数$w_{0}=-\\theta$, 这样$z$可以用更加紧凑形式表示$z=w_{0}+w_{1}x_{1}+...+w_{m}x_{m}=w^{T}x$，此时\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/10/575a70ea7e4fb.png)\n",
    "\n",
    "下面左图描述了感知机的激活函数怎样将网络输入$z=w^{T}x$压缩到二元输出(-1,1)，右图描述了感知机如何区分两个线性可分的类别。\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/10/575a71ba0e84b.png)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "不论MCP神经元还是Rosenblatt的阈值感知机模型，他们背后的idea都是试图使用简单的方法来模拟大脑中单个神经元的工作方式：要么传递信号要么不传递。因此，Rosenblatt最初的感知机规则非常简单，步骤如下：\n",
    "* 1. 将权重参数初始化为0或者很小的随机数。\n",
    "* 2. 对于每一个训练集样本$x^{(i)}$,执行下面的步骤：\n",
    "* * 1、计算输出值 $\\hat{y}$.\n",
    "* * 2、更新权重参数.\n",
    "\n",
    "此处的输出值就是单位阶跃函数预测的类别(1,-1)，参数向量$w$中的每个$w_{j}$的更新过程可以用数学语言表示为：\n",
    "\n",
    "$$w_{j}:= w_{j} + \\Delta w_{j}$$\n",
    "\n",
    "其中$\\Delta w_{j}$，用于更新权重$w_{j}$,在感知机算法中的计算公式为:\n",
    "\n",
    "$$\\Delta w_{j} = \\eta (y^{(i)} - {\\hat{y}}^{(i)})x_{j}^{(i)}$$\n",
    "\n",
    "其中$\\eta$ 称为学习率(learning rate), 是一个介于0.0和1.0之间的常数，$y^{(i)}$是第$i$个训练样本的真实类别，$\\hat{y}^{(i)}$是对第$i$个训练样本的预测类别。 通俗地说，我们在计算出一个$\\hat{y}^{(i)}$后,就能计算出样本$i$对应的所有的的$\\Delta w_{j}$，然后同时更新w中的每一个权重参数；然后不断重复上面的步骤。\n",
    "\n",
    "举个例子，如果$x$的特征维度是2，我们可以将更新过程写为：\n",
    "\n",
    "$$\\Delta w_{0} = \\eta(y^{(i)}-{\\hat{y}}^{(i)})$$\n",
    "\n",
    "$$\\Delta w_{1} = \\eta(y^{(i)}-{\\hat{y}}^{(i)})x_{1}^{(i)}$$\n",
    "\n",
    "$$\\Delta w_{2} = \\eta(y^{(i)}-{\\hat{y}}^{(i)})x_{2}^{(i)}$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "\n",
    "在我们用Python实现感知机算法之前，我们先来看一下这个简单的算法是多么神奇。如果感知机预测的类别正确，则权重参数不做改变，因为：\n",
    "\n",
    "$$\\Delta w_{j}=\\eta(-1-(-1))x_{j}^{(i)}=0$$\n",
    "\n",
    "$$\\Delta w_{j}=\\eta(1-1)x_{j}^{(i)}=0$$\n",
    "\n",
    "当预测结果不正确时，权重会朝着正确类别方向更新(如果正确类别是1，权重参数会增大；如果正确类别是-1，权重参数会减小):\n",
    "\n",
    "$$\\Delta w_{j}=\\eta(1-(-1))x_{j}^{(i)}=\\eta (2)x_{j}^{(i)}$$\n",
    "\n",
    "$$\\Delta w_{j}=\\eta(-1-1)x_{j}^{(i)}=\\eta (-2)x_{j}^{(i)}$$\n",
    "\n",
    "我们可以具体化上面的乘数$x_{j}^{(i)}$,比如一个简单的例子：\n",
    "\n",
    "$$\\hat {y}_{j}^{(i)}=+1, y^{(i)}=-1, \\eta =1$$\n",
    "\n",
    "假设$x_{j}^{(i)}=0.5$,我们将样本误分类为-1.此时，更新过程会对权重参数加1，下一次在对样本i计算输出值时，有更大的可能输出1：\n",
    "\n",
    "$$\\Delta w_{j}^{(i)} = (1-(-1))\\cdot 0.5=2 \\times 0.5=1$$\n",
    "\n",
    "参数更新$\\Delta w_{j}$和样本$x_{j}^{(i)}$成正比。比如，如果我们有另一个样本$x_{j}^{(i)}$=2 被误分类为-1，在更新$w_{j}$时会朝着正确方向更新更多(相比较$x_{j}^{(i)}$=0.5的情况)：\n",
    "\n",
    "$$\\Delta w_{j}^{(i)} = (1-(-1))\\cdot 2=2\\times 2=4$$\n",
    "\n",
    "**感知机算法仅在两个类别确实线性可分并且学习率充分小的情况下才能保证收敛**。如果两个类别不能被一个线性超平面分开，我们可以通过设置迭代次数(epoch)来停止算法的学习过程。\n",
    "\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/10/575a7bfd8564d.png)\n",
    "\n",
    "在进入下一节的代码实现之前，我们来总结一下感知机的要点：\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/10/575a7c3faa0db.png)\n",
    "\n",
    "感知机接收一个样本输入x，然后将其和权重w结合，计算网络输入z。z接着被传递给激活函数，产生一个二分类输出-1或1作为预测的样本类别。在整个学习阶段，输出用于计算预测错误率(y-$\\hat{y}$)和更新权重参数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 使用Python实现感知机算法\n",
    "\n",
    "在前一节，我们学习了Rosenblatt感知机算法；这一节我们用Python对其进行编码实现，并且应用于Iris数据集。关于代码的实现，我们使用面向对象的编程思想，定义一个感知机算法类，类中的方法主要有初始化方法，训练(fit)方法和预测(predict)方法。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "class Perceptron(object):\n",
    "    \"\"\"Perceptron classifier.\n",
    "\n",
    "    Parameters\n",
    "    ------------\n",
    "    eta:float\n",
    "    Learning rate (between 0.0 and 1.0)\n",
    "    n_iter:int\n",
    "    Passes over the training dataset.\n",
    "\n",
    "    Attributes\n",
    "    -------------\n",
    "    w_: 1d-array\n",
    "    Weights after fitting.\n",
    "    errors_: list\n",
    "    Numebr of misclassifications in every epoch.\n",
    "\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self, eta=0.01, n_iter=10):\n",
    "        self.eta = eta\n",
    "        self.n_iter = n_iter\n",
    "\n",
    "    def fit(self, X, y):\n",
    "        \"\"\"Fit training data.\n",
    "\n",
    "        Parameters\n",
    "        ------------\n",
    "        X: {array-like}, shape=[n_samples, n_features]\n",
    "        Training vectors, where n_samples is the number of samples\n",
    "        and n_featuers is the number of features.\n",
    "        y: array-like, shape=[n_smaples]\n",
    "        Target values.\n",
    "\n",
    "        Returns\n",
    "        ----------\n",
    "        self: object\n",
    "        \"\"\"\n",
    "\n",
    "        self.w_ = np.zeros(1 + X.shape[1]) # Add w_0\n",
    "        self.errors_ = []\n",
    "\n",
    "        for _ in range(self.n_iter):\n",
    "            errors = 0\n",
    "            for xi, target in zip(X, y):\n",
    "                update = self.eta * (target - self.predict(xi))\n",
    "                self.w_[1:] += update * xi\n",
    "                self.w_[0] += update\n",
    "                errors += int(update != 0.0)\n",
    "            self.errors_.append(errors)\n",
    "        return self\n",
    "\n",
    "    def net_input(self, X):\n",
    "        \"\"\"Calculate net input\"\"\"\n",
    "        return np.dot(X, self.w_[1:]) + self.w_[0]\n",
    "\n",
    "    def predict(self, X):\n",
    "        \"\"\"Return class label after unit step\"\"\"\n",
    "        return np.where(self.net_input(X) >= 0.0, 1, -1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们初始化一个Perceptron对象，并且对学习率eta和迭代次数n\\_iter赋值，fit方法先对权重参数初始化，然后遍历训练集中每一个样本，根据感知机算法对权重进行更新。类别通过predict方法进行预测。除此之外，self.errors_ 还记录了每一轮中误分类的样本数，有助于接下来我们分析感知机的训练过程。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 基于Iris数据集训练感知机模型\n",
    "\n",
    "我们使用Iris数据集检验上面的感知机代码，由于我们实现的是一个二分类感知机算法，所以我们仅使用Iris中Setosa和Versicolor两种花的数据。为了简便，我们仅使用sepal length和petal length两维度的特征。记住，感知机模型不局限于二分类问题，可以用通过One-vs-All技巧扩展到多分类问题。\n",
    "\n",
    "**One-vs-All(OvA)**有时也被称为**One-vs-Rest(OvR)**，是一种常用的将二分类分类器扩展为多分类分类器的技巧。通过OvA技巧，我们为每一个类别训练一个分类器，此时，对应类别为正类，其余所有类别为负类。对新样本数据进行类别预测时，我们使用训练好的所有类别模型对其预测，将具有最高置信度的类别作为最后的结果。\n",
    "\n",
    "\n",
    "回到刚才的Iris数据集，我们使用pandas读取数据，然后通过pandas中的tail方法输出最后五行数据，看一下Iris数据集格式:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data', header=None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>145</th>\n",
       "      <td>6.7</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.2</td>\n",
       "      <td>2.3</td>\n",
       "      <td>Iris-virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>146</th>\n",
       "      <td>6.3</td>\n",
       "      <td>2.5</td>\n",
       "      <td>5.0</td>\n",
       "      <td>1.9</td>\n",
       "      <td>Iris-virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>147</th>\n",
       "      <td>6.5</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.2</td>\n",
       "      <td>2.0</td>\n",
       "      <td>Iris-virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>148</th>\n",
       "      <td>6.2</td>\n",
       "      <td>3.4</td>\n",
       "      <td>5.4</td>\n",
       "      <td>2.3</td>\n",
       "      <td>Iris-virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>149</th>\n",
       "      <td>5.9</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.1</td>\n",
       "      <td>1.8</td>\n",
       "      <td>Iris-virginica</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       0    1    2    3               4\n",
       "145  6.7  3.0  5.2  2.3  Iris-virginica\n",
       "146  6.3  2.5  5.0  1.9  Iris-virginica\n",
       "147  6.5  3.0  5.2  2.0  Iris-virginica\n",
       "148  6.2  3.4  5.4  2.3  Iris-virginica\n",
       "149  5.9  3.0  5.1  1.8  Iris-virginica"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来我们抽取出前100条样本，这正好是Setosa和Versicolor对应的样本，我们将Versicolor对应的数据作为类别1，Setosa对应的作为-1。对于特征，我们抽取出sepal length和petal length两维度特征，然后用散点图对数据进行可视化:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = df.iloc[0:100, 4].values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = np.where(y == 'Iris-setosa', -1, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = df.iloc[0:100, [0, 2]].values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEKCAYAAAARnO4WAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3XuYHHWd7/H3NzdDEEFJHg8kSwY9Eg2ZzOQiGogm3FVYlueBmF3CJREeDmQlQZQFL2tmWaLHxSMHj6gbbmFNlotzYI9yQAUEQUVIJk4SAstFSSCBIwmYGAxBkvmeP6o76ZnMpaq7f91VXZ/X89QzU9U11d+qnnzT8/3271fm7oiISOMbVO8ARESkNpTwRURyQglfRCQnlPBFRHJCCV9EJCeU8EVEckIJX0QkJ5TwRURyQglfRCQnhtQ7gFIjR470pqameochIpIZHR0dW9x9VJx9U5Xwm5qaWLlyZb3DEBHJDDPbEHdflXRERHJCCV9EJCeU8EVEciJVNfzevP3222zcuJGdO3fWO5TMGz58OGPGjGHo0KH1DkVE6iD1CX/jxo0ccMABNDU1YWb1Diez3J3XXnuNjRs3cvjhh9c7HBGpg9SXdHbu3MnBBx+sZF8hM+Pggw/WX0oNpOe9i3QvIxlI6hM+oGRfJbqOjaOtDT73ub1J3j1ab2urZ1SSdplI+CKylzts3QrXXbc36X/uc9H61q16py99S30NP0uWLl3KSSedxKGHHlrvUKSBmcG110bfX3ddtAAsXBht1x9y0he9w6+ipUuX8vLLL9c7DMmB0qRfpGQvAwma8M1svZmtNbNOM6vNnAnLl0NTEwwaFH1dvryiw/35z3/mlFNOoaWlhQkTJnDHHXfQ0dHBjBkzmDJlCieffDKvvPIK7e3trFy5kjlz5tDa2sqbb77Jgw8+yKRJk2hubuYzn/kMb731FgBXXnkl48ePZ+LEiXzhC18A4Mc//jEf+chHmDRpEieccAJ/+MMfKrwQ0siKZZxSpTV96Vuum93uHmwB1gMj4+4/ZcoU7+mpp57aZ1ufli1zHzHCPXoNo2XEiGh7mdrb2/2CCy7Ys75161afNm2av/rqq+7ufvvtt/u8efPc3X3GjBm+YsUKd3d/8803fcyYMf7MM8+4u/s555zj1157rW/ZssWPOOII7+rqcnf3P/7xj+7u/vrrr+/ZdsMNN/hll11Wdsz9SXQ9JZW6utwXLox+vRcu7H1derdoUfdrVLx2ixbVM6rKACs9Zo5trJLOl78MO3Z037ZjR7S9TM3Nzdx///1cccUVPProo7z00ks8+eSTnHjiibS2tnL11VezcePGfX7umWee4fDDD+eII44A4LzzzuORRx7hwAMPZPjw4Zx//vncddddjBgxAojGG5x88sk0NzdzzTXXsG7durJjlsZmBgcd1L1mf+210fpBB6ms0xc1u8M3bR34mZk58K/uviTos734YrLtMRxxxBGsWrWKe++9l6985Sscd9xxHHnkkTz22GNlHW/IkCE88cQTPPjgg7S3t/Od73yHn//851xyySVcdtllnHbaaTz88MO06fN10o+2tihBFZN7Mekr2fdNze7wTdvp7j4Z+CTw92b28Z47mNmFZrbSzFZu3ry5smc77LBk22N4+eWXGTFiBGeffTaXX345jz/+OJs3b96T8N9+++0978YPOOAAtm/fDsC4ceNYv349zz//PAA/+MEPmDFjBm+88Qbbtm3jU5/6FNdeey2rV68GYNu2bYwePRqAW2+9tex4JT96Jqh6JKys1cNDNruzcC2CJnx331T4+ipwN3BUL/sscfep7j511KhYc/j3bfFiKJRI9hgxItpeprVr13LUUUfR2trKP/3TP3HVVVfR3t7OFVdcQUtLC62trfz6178GYO7cuVx00UW0trbi7txyyy3MmjWL5uZmBg0axEUXXcT27ds59dRTmThxItOnT+db3/oWAG1tbcyaNYspU6YwcuTIsuMVqZUsDv4K1ezOzLWIW+xPugD7AweUfP9r4BP9/UzFTVv3qEE7dqy7WfS1goZtI1LTVqohi43jUDHX+1qQoGkbsob/XuDuwnD+IcC/u/tPAj5fZM6caBGRYLJYD++r2Q2VNbuzdC3MU1Romjp1qve8xeHTTz/Nhz70oTpF1Hh0PaWa3KMhL0VdXelKcL0pbXb3tl7JcetxLcysw92nxtm3sT6WKZJxWWj8FWV18FeSZnfc1yMr10IJXyQlMtP4o/tn2BcujN7NLlzY/TPuWRf39cjStdDkaSIpUDooCKLab2kSqVbZoVpC1cPTIsnrkaVroRp+zuh6plfpO8WiNDb+SoWqh6dB0tejXtdCNfyU++pXv8oDDzyQ+OcefvhhTj311AARSRpkcQbMNAz+CiXp65GFa9FwCT8tTS93p6urq9fHrrrqKk444YTgMezatSv4c0j1hGz89fxV7ONXM7G0/HtLIg2N2Hpdt4ZK+CGaXldeeSXXX399yXO08c1vfpNrrrmGD3/4w0ycOJFFixYBsH79esaNG8e5557LhAkTeOmll5g7dy4TJkygubmZawtvF+bOnUt7ezsAK1as4Oijj6alpYWjjjqK7du3s3PnTubNm0dzczOTJk3ioYce2ieu119/ndNPP52JEyfy0Y9+lDVr1uyJ75xzzuGYY47hnHPOKf/EpaZCNv5mzoQpU/Ym+a6uaH3mzMpizlKTuSgNjdh6XreGSfihZsKbPXs2d9555571O++8k1GjRvHcc8/xxBNP0NnZSUdHB4888ggAzz33HPPnz2fdunVs2bKFTZs28eSTT7J27VrmzZvX7dh/+ctfmD17Ntdddx2rV6/mgQceYL/99uP666/HzFi7di233XYb55133j43H1+0aBGTJk1izZo1fO1rX+Pcc8/d89hTTz3FAw88wG233VbeSUvNhZoBs6sLtm2Dzs69SX/KlGh927by3+lncebJJDGHej3qft3iDsmtxVLp1AqlQ5qLSzWGNn/wgx/0TZs2eWdnpx999NH++c9/3seOHestLS3e0tLi73//+/3GG2/0F154wZuamvb83Ouvv+7ve9/7/LOf/azfd999vnv3bnd3P++88/yHP/yhr1mzxo8++uh9nu/000/3Bx98cM/69OnTffXq1f7QQw/5Kaec4u7ura2t/rvf/W7PPmPGjPFt27b5okWLvK2trc9z0dQK6dbzd7Uaw/J373Zvbe3+76K1NdpeiVD/3kJKGnOI16Pa1428zocfquk1a9Ys2tvbueOOO5g9ezbuzhe/+EU6Ozvp7Ozk+eef5/zzzwdg//333/Nz7373u1m9ejUzZ87k+9//PhdccEFlgcRUGoNkS4jG36BB0NHRfVtHR/dRoeXIapO53o3Yel63hkr4oZoss2fP5vbbb6e9vZ1Zs2Zx8sknc/PNN/PGG28AsGnTJl599dV9fm7Lli10dXVxxhlncPXVV7Nq1apuj48bN45XXnmFFStWALB9+3Z27drFxz72MZYXbs347LPP8uKLLzJu3LhuP1u6z8MPP8zIkSN517veVdmJSkPq6oLJk7tvmzy58sZt6NGlSRqbaWjExlXPGBom4Ydsshx55JFs376d0aNHc8ghh3DSSSdx1llnMW3aNJqbmznzzDP3zINfatOmTcycOZPW1lbOPvtsvv71r3d7fNiwYdxxxx1ccskltLS0cOKJJ7Jz507mz59PV1cXzc3NzJ49m6VLl/KOd7yj28+2tbXR0dHBxIkTufLKKzWHvvSqqwsOPRRWr4aWFti9O/q6enW0vZIafsjRpUkam2loxMZV9xji1n5qsVRaw2/E+1VWm2r4+dLV5T56dFQnXrAgWl+wIFofPbqymnSof29JphtOOjVxGnJEtWMgQQ2/7km+dKnGfPghmiyNRAk/f0qTfHEpJv9qHLu/9UqOG7exmYZGbFLVjCFJwtfUCjmj65lPntFpjOPGnMXzq5aGm1ohTf8pZVkjXMeep1CtU0py3FAxJBV39Gyxblyqv3pxWq5x3JiTnl+epT7hDx8+nNdee60hklU9uTuvvfYaw4cPr3coZQs1QjFEgzC0uKNni/HFbRKm4RoniTnp+eVd6qdHHjNmDBs3bmTz5s31DiXzhg8fzpgxY+odRlncw0wfnOS4oWJIqufo2Y6OvaNnW1ujx4vljSRT96bhGieNOUtTE6dC3GJ/LZbemrYiRaFGdoZsEIaSdPRs3CZhGq5x0piT7ttoaKSmrUippI28uPOTZ7FB2NUFgwfvXd+9u/LRs5COa5xU0jgaaQ7/hmvaikCy5lw5NeM4x01Lg7BYsy9VWtMvVxqucVJZ7MHUTdw/BWqxqKQjfQk1GCfkIJ9QSss5xTJOz/VyZPG6pSWOeqKRBl6JFCUZoZikZpzkuGkYqenuPmNG9+ReTPozZlR23DRc46Sy2IOppiQJXzV8yRQPWJfPWg249NM4va2XKw3XOKmkcaShB1MtquFLw4o7Xa0nrBknmQY3Lfcu7Zncq5Hsk0h6jdMQR1pirpu4fwrUYlFJR6qhUWu1tRC39JKWSctUw09W0kn9wCuRpDQYpzyeYIBUGgZ0JY1Dvxeohi+NKy219iwpljyKyRm6J8je9o9zjZMet5y4s9aDqZYkNXwlfBHpJlRTs9GapWmhpq1IQj3f9/T3PijuLJWh4wghVFMz983SlFDCl9xLMvoy7iyVoeMIobTsUs2ZJ0MdV5JTwpdcK20oFpNPMTlt3do9GfWcpbKY7Ds7o+2VvNNPEkcofTU1Fy6srKkZ6riSnGr4kntJGoqlSb6otTWaorjSz8GHbmwmiSNEU7PRmqVpoaatSEJJGoqhZqlMGocIqGkrkog7XHpp922XXtp7GaWrCyZP7r5t8uT+by/Y33rPx9TYlJCU8CXX3GHaNPj2t2HBgihxL1gQrU+btm8N/9BDYfVqaGmJ3tm3tETrhx66b9IPdVs/kXIp4YvEZAZDCmPTZ8yI1mfMiNaHDOl7dGmcJqwam1ITcedgqMWiuXSkHrq63Bcs8G5T5i5Y0Pf0ukn2DXlbPxH3ZHPpqGkrQrjpddWEldBS1bQ1s8Fm9lszuyf0c4mUijsiNkmzNOm+cZvB5UjaEI67rzSuWtTwFwJP1+B5RPaIOyI2SbM06b5xm8Hl0H1cpRxBE76ZjQFOAW4M+TwipZKMiE3SLE1LYzVJQzgNI3glReIW+8tZgHZgCjATuGeg/dW0lWopvbF3cenvBt9JmqVx903S4E0q7/dxlb1IQ9PWzE4FPuXu881sJvAFdz+1l/0uBC4EOOyww6Zs2LAhSDxSO56SIfRJRsSGijkto3LVPG5caWnaHgOcZmbrgduB48xsWc+d3H2Ju09196mjRo0KGI7UQlrqxcUyTqnSmn6pUDEvWtR7DIsWVXZcCNdolgYX90+BShZU0smFtNwztLScUyzj9FwPHXOSGJLSfVylFLqnrdRD6T1Cr7tu76yPtZ7xcdAgOPDA7rNYdnRE764PPLB7aSNUzIMGwWmnRd93du4t67S2RtsrKevoPq5SLg28kqpLS724q2vfOPqr4YeIOXQNP6/3cZW90lLDlxxKU724Z0LrLxmGuq3fZZd133bZZdW7FnHPL+m+0riU8KVqiokzDTM+xm3Ehoo5TddCpEg1fKmatNSLvWSwEUQxlCbf0nJGqJjTci1ESqmGL1WXhnpx6Tvsov4asaFiTsO1kMamWxyKkJ7msUhIatpK7nngmSpFskgJXxqOB56pUiSrlPBFRHJCCV8ajhk89tjed/WDBu19t//YY6rjS36paSsNS01byQM1bSX3ko6e7bk9Re+DRKpmwIRvZseY2f1m9qyZ/d7MXjCz39ciOJFyJB3lmpYpnUVCizPS9ibgc0AHsDtsOCKVSzLKNcmoXJGsG7CGb2aPu/tHahGMavhSTXFHuSYdlSuSJlUZaWtmkwvffhoYDNwFvFV83N1XVRjnPpTwpV7U4JWsSpLw+yvp/I8e66UHdOC4pIFJumiel0hfDd5az7sjElqfCd/djwUws/e5e7cmrZm9L3RgElZbW1S7Lia1YtI76KB8NSt7NnhLa/iwb9LXdZMsi/OxzPZetv2w2oFI7ZQ2KoufTikmua1b8/WRxL4avAsX9t/gzft1k2zqr4b/QeBI4F+Ay0seehdwubsfWe1gVMOvHTUqu1ODV7KqWk3bvwFOB04DflTy0Hbgdnf/daWB9qSEX1tqVJZH103SpCojbd39/7j7POBUd59XsiwIkeylttJ079ks0XWTLItTwz/LzL7dY/nnwl8AkkG632p5dN0k6+KMtH0H8EH2NmrPAF4AWszsWHe/tM+flFTS/VbLo+smWRdnpO1vgGPcfXdhfQjwKDAdWOvu46sVjGr4taXPk5dH103SpNqzZb4beGfJ+v7Aewr/AbzV+49IvWjWx/B6Jncle8mKOAn/X4BOM7vFzJYCvwWuMbP9gQdCBifJJJn1UTNEiuTPgAnf3W8Cjgb+A7gbmO7uN7r7n9398v5/WmolyaAgDSASyadYd7wys9HAWEqavO7+SLWDUQ2/MkkGBWkAkUhjqMrAq5KDfQOYDawDugqb3d1PqyjKXijhVy7JoCANIBLJvmo3bU8Hxrn7Ke7+14Wl6sleKpdkUJAGEInkT5yE/3tgaOhApDJJBgVpAJFIPsUZeLWD6FM6D9L9BigLgkUliSUZFKQBRCL5FKeGf15v29391moHoxp+5ZIMCtIAIpHsq9Ydr4AosZvZfsBh7v5MxdFJUEkGBWkAkUi+DFjDN7O/BjqBnxTWW83sR/3/lIiIpE2cpm0bcBSwFcDdOwHd4lBEJGPiJPy33X1bj21dve4pIiKpFedTOuvM7CxgsJl9AFgA6AYoIiIZE+cd/iVE97Z9C7gN+BOgOfBzRrNwimRfnE/p7AC+XFhiM7PhwCNEN1AZArS7+6JygpT6amuLJlUrfma/OHDroIM0u6ZIlvSZ8M3sx0Cf7+NiTK/wFnCcu79hZkOBX5rZfe7+m/JClXoonVkToqRfOkpXn90XyY7+3uF/s5IDezSi643C6tDCokJAxpSOwr3uur2JXzNrimRPrOmRyz642WCgA/ivwPXufkUv+1wIXAhw2GGHTdmwYUOweKR8mllTJJ2qPVtm2dx9t7u3AmOAo8xsQi/7LHH3qe4+ddSoUSHDkTJpZk2RxhA04Re5+1bgIeATtXg+qR7NrCnSOOJ8Dr8sZjaKaNDW1sJcPCcC3wj1fBKGZtYUaRwhP6VzCHBroY4/CLjT3e8pK0qpq7a27p/GKSZ9JXuRbAn5KZ01wKRKjiHpoZk1RbKvz4Tv7r+oZSAiIhLWgDX8wvw5XwfGA8OL291dM2aKiGRInE/p3AJ8D9gFHAv8G7AsZFAiIlJ9cRL+fu7+INEgrQ3u3gacEjYsERGptjgfy3zLzAYBz5nZZ4FNwDvDhiUiItUW5x3+QmAE0Tz4U4BzgF5vbC4iIukVZ3rkFQCFd/kL3H178KhERKTq4tzEfKqZrQXWAGvNbLWZTQkfmoiIVFOcGv7NwHx3fxTAzKYTfXJnYsjARESkuuLU8HcXkz2Au/+S6COaIiKSIXHe4f/CzP6V6H62DswGHjazyQDuvipgfCIiUiVxEn5L4WvP+9FOIvoP4LiqRiQiIkHE+ZTOsbUIREREworzKZ33mtlNZnZfYX28mZ0fPjQREammOE3bpcBPgUML688Cl4YKSEREwoiT8Ee6+51AF4C77wJ2B41KRESqLk7C/7OZHUzh7ldm9lFgW9CoRESk6uJ8Sucy4EfA+83sV8Ao4MygUYmISNXF+ZTOKjObAYwDDHjG3d8OHpmIiFRVnE/pzCKaE38dcDpwR3HQlYiIZEecGv4/uvv2whw6xwM3Ed0BS0REMiTWXDqFr6cAN7j7/wWGhQtJRERCiJPwNxXm0pkN3Gtm74j5cyIikiJxEveniQZenezuW4H3AJcHjUpERKouzqd0dgB3lay/ArwSMigREak+lWZERHJCCV9EJCeU8EVEckIJX0QkJ5TwRURyQglfRCQnlPBFRHJCCV9EJCeU8EVEckIJX0QkJ5TwRURyQglfRCQnlPBFRHJCCV9EJCeCJXwz+ysze8jMnjKzdWa2MNRziYjIwAacD78Cu4DPu/sqMzsA6DCz+939qYDPKSIifQj2Dt/dX3H3VYXvtwNPA6NDPZ+IiPSvJjV8M2sCJgGP9/LYhWa20sxWbt68uRbhiIjkUvCEb2bvBP43cKm7/6nn4+6+xN2nuvvUUaNGhQ5HRCS3giZ8MxtKlOyXu/tdA+0vfVi+HJqaYNCg6Ovy5fmMQUQqEqxpa2YG3AQ87e7fCvU8DW/5crjwQtixI1rfsCFaB5gzJz8xiEjFzN3DHNhsOvAosBboKmz+krvf29fPTJ061VeuXBkknsxqaooSbE9jx8L69fmJQUR6ZWYd7j41zr7B3uG7+y8BC3X83HjxxWTbGzUGEamYRtqm3WGHJdveqDGISMWU8NNu8WIYMaL7thEjou15ikFEKqaEn3Zz5sCSJVG93Cz6umRJbZulaYhBRCoWrGlbDjVtRUSSSdK01Tt8EZGcUMIXEckJJXyJJy0jbefPhyFDol7CkCHReq2l5VqIJBRyemRpFGkZaTt/Pnzve3vXd+/eu/7d79YmhrRcC5EyqGkrA0vLSNshQ6Ik39PgwbBrV21iSMu1EClQ01aqKy0jbXtL9v1tDyEt10KkDEr4MrC0jLQdPDjZ9hDSci1EyqCEXw9Jm36hGpUnnBAds7iccELv+y1eDEOHdt82dGjtR9oWa+Vxt4egUceSZe6emmXKlCne8JYtcx8xwh32LiNGRNt7c/HF3fctLhdfXFkcxx/f+3GPP773mIcN677fsGF9xxzSxRe7Dx4cxTB4cOXXoRzLlrmPHetuFn2tx3UQKQBWeswcq6ZtrSVt+oVqVFo/E5n2/J1Qo1IktdS0TbOkTT81KkWkSpTwqyVuXT5p0y9pozJuXT6JpDEn7Tkk6Wlo4JVI+eLWfmqxZLaGn6QuH7KGn6QuP3587/uOH1/Zvkl7DkmuR6h+RhJJXz+RwEhQw697ki9dMpvwx47tPRGNHdv7/kmbfnEblb3FUFwqiTnJcYtx9lwGD+495iRxJD12CElfa5HAkiR8NW2rYdCgfRudEJUdurr23R5KkkZskpiTHDfJviHjCCUtr7VIgZq2tZbFwTihYk7ac0gShwZeiVQkfwk/RMNt8eLoeKUGDareYJy4jcrjj4+/ffHifRPl4MG9x5zkuEkHRyUZyJT02KFeaw28kqyKW/upxRK8hh+q4RaymZjk2MuW9b5vpQ3QpDH0rLUPHtz/NU7S04jbzwjZXNXAK0kRVMPvQ6gBRCFncUxy7CTnl+S4oWIIKS1xiASWpIafr4QfquEWspmYtUZsWpqaaYlDJDA1bfuSlkZlqGOHaoCGiiGktMQhkiL5SvhJm6txm34hm4lJjh2qARoqhnLEvXZpmeFTJE3iFvtrsQRv2iZtPiYdERuqmZhkhsi4DcWkzdUQMSSVdERzWmb4FAkINW37kIbmY1qaiWmJI4kkMWfx/ETKoKZtX9LQfExLMzEtcSSRJOYsnp9IGdS07Usamo9paSamJY4kksScxfMTCSxfCT8Nzce0jNRcvBiGDeu+bdiwdDc1k1y7tFxnTaUsaRK32F+LpSazZaal+VjvkZrLlrkPHdq9qTl0aPqbmkmuXb2vs6ZSlhpATVsZkJqa4ekaSw2ohi8D020Lw9M1lpRpjISvOmlyamqGp2ssKZP9hL98edR03bAhqpJu2BCtK+n3Ly1NzUamaywpk/2E/+Uvw44d3bft2BFtl77NmQNLlkT1ZLPo65Il0XapDl1jSZnsN201wEZEcixfTVvVSUVEYgmW8M3sZjN71cyeDPUcQHrqpGoci0jKhXyHvxT4RMDjR9JQJ1XjWEQyIGgN38yagHvcfUKc/TM78EoDbESkTjJVwzezC81spZmt3Lx5c73DKY8G2IhIBtQ94bv7Enef6u5TR40aVe9wyqPGsYhkQN0TfkNIS+NYRKQfSvjVkIbGsYjIAIaEOrCZ3QbMBEaa2UZgkbvfFOr56m7OHCV4EUm1YAnf3f8u1LFFRCQ5lXRERHJCCV9EJCeU8EVEckIJX0QkJ5TwRURyIlXz4ZvZZqCXSWnqbiSwpd5BBKTzyzadX3ZV49zGunusaQpSlfDTysxWxp2cKIt0ftmm88uuWp+bSjoiIjmhhC8ikhNK+PEsqXcAgen8sk3nl101PTfV8EVEckLv8EVEckIJvwczG2xmvzWze3p5bK6ZbTazzsJyQT1iLJeZrTeztYXY97mXpEW+bWbPm9kaM5tcjzjLFeP8ZprZtpLX76v1iLNcZnaQmbWb2X+a2dNmNq3H45l9/WKcW2ZfOzMbVxJ3p5n9ycwu7bFPTV67YLNlZthC4GngXX08foe7f7aG8VTbse7e1+d+Pwl8oLB8BPhe4WuW9Hd+AI+6+6k1i6a6rgN+4u5nmtkwoMdddzL9+g10bpDR187dnwFaIXpDCWwC7u6xW01eO73DL2FmY4BTgBvrHUud/A3wbx75DXCQmR1S76AEzOxA4OPATQDu/hd339pjt0y+fjHPrVEcD/zO3XsOMK3Ja6eE393/BP4B6OpnnzMKf3K1m9lf1SiuanHgZ2bWYWYX9vL4aOClkvWNhW1ZMdD5AUwzs9Vmdp+ZHVnL4Cp0OLAZuKVQcrzRzPbvsU9WX7845wbZfe1K/S1wWy/ba/LaKeEXmNmpwKvu3tHPbj8Gmtx9InA/cGtNgque6e4+mejPx783s4/XO6AqG+j8VhENQ28B/hfwH7UOsAJDgMnA99x9EvBn4Mr6hlQ1cc4ty68dAIVS1WnAD+sVgxL+XscAp5nZeuB24DgzW1a6g7u/5u5vFVZvBKbUNsTKuPumwtdXiWqIR/XYZRNQ+lfLmMK2TBjo/Nz9T+7+RuH7e4GhZjay5oGWZyOw0d0fL6y3EyXJUll9/QY8t4y/dkWfBFa5+x96eawmr50SfoG7f9Hdx7h7E9GfXT9397NL9+lRUzuNqLmbCWa2v5kdUPweOAl4ssduPwJ6lTiGAAADTUlEQVTOLXxi4KPANnd/pcahliXO+ZnZfzEzK3x/FNHv/2u1jrUc7v7/gJfMbFxh0/HAUz12y+TrF+fcsvzalfg7ei/nQI1eO31KZwBmdhWw0t1/BCwws9OAXcDrwNx6xpbQe4G7C/9mhgD/7u4/MbOLANz9+8C9wKeA54EdwLw6xVqOOOd3JnCxme0C3gT+1rM18vASYHmhNPB7YF4DvX4DnVumX7vCm5ATgf9Wsq3mr51G2oqI5IRKOiIiOaGELyKSE0r4IiI5oYQvIpITSvgiIjmhhC8Nz6JZTg+Nsd9SMzsz7vYqxPWlku+bzKznuAiRqlLClzyYCwyY8OvgSwPvIlI9SviSKYV3wv9pZssL86a3m9mIwmNTzOwXhcnTfmpmhxTemU8lGtTTaWb7mdlXzWyFmT1pZkuKIzhjPv8+z1HY/rCZfcPMnjCzZ83sY4XtI8zsTjN7yszuNrPHzWyqmf13YL9CTMsLhx9sZjeY2Toz+5mZ7Vfdqyd5p4QvWTQO+K67fwj4EzDfzIYSTap1prtPAW4GFrt7O7ASmOPure7+JvAdd/+wu08A9gNizbHe13OU7DLE3Y8CLgUWFbbNB/7o7uOBf6Qw/5K7Xwm8WYhpTmHfDwDXu/uRwFbgjOSXRqRvmlpBsugld/9V4ftlwALgJ8AE4P7CG/bBQF9zkRxrZv9AdJON9wDriGZCHci4AZ7jrsLXDqCp8P10opt74O5Pmtmafo7/grt39nIMkapQwpcs6jkfiAMGrHP3ab3sv4eZDQe+C0x195fMrA0YHvN5B3qO4kyquynv39ZbJd/vJvrrQ6RqVNKRLDrM9t7z9Czgl8AzwKjidjMbWnKTjO3AAYXvi8l9i5m9k2hSrrj6e46+/Ar4dGH/8UBzyWNvF8pEIjWhhC9Z9AzRDU6eBt5NdOOMvxAl72+Y2WqgEzi6sP9S4Ptm1kn0LvoGoqmTfwqsiPukAzxHX75L9J/EU8DVROWjbYXHlgBrSpq2IkFptkzJFDNrAu4pNFxTz6KbVg91951m9n7gAWBc4T8PkZpSDV8krBHAQ4XSjQHzleylXvQOX0QkJ1TDFxHJCSV8EZGcUMIXEckJJXwRkZxQwhcRyQklfBGRnPj//yXvQ/ApVCwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f515292d9d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(X[:50, 0], X[:50, 1], color='red', marker='o', label='setosa')\n",
    "plt.scatter(X[50:100, 0], X[50:100, 1], color='blue', marker='x', label='versicolor')\n",
    "plt.xlabel('petal length')\n",
    "plt.ylabel('sepal length')\n",
    "plt.legend(loc='upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在开始训练我们的感知机模型，为了更好地了解感知机训练过程，我们将每一轮的误分类数目可视化出来，检查算法是否收敛和找到分界线："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "ppn = Perceptron(eta=0.1, n_iter=10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<__main__.Perceptron at 0x7f5152922c50>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ppn.fit(X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3Xl8VfWd//HXJwsQwhJ22cOSYLUuaERcUFax2lFrtbVTWxesy7ggMs7oPH7TTjudTjtWrFurVKloq7a1lLGOFsMi7mhAEVAJYROCSADDGiCEz++PexIDZjlA7j03ue/n43Ee995zzz3nnavkk+/5nvP9mrsjIiICkBZ1ABERSR4qCiIiUkNFQUREaqgoiIhIDRUFERGpoaIgIiI1VBRERKSGioKIiNRQURARkRoZUQc4XF27dvXc3NyoY4iINCsLFy7c7O7dGtuu2RWF3NxcioqKoo4hItKsmNnaMNvp9JGIiNRQURARkRoqCiIiUkNFQUREaqgoiIhIjbhdfWRmbYBXgdbBcZ5z9x8dsk1r4EngVGAL8G13XxOvTBIz871S7pm1nA3lFfTKyeLO8UO4ZGjvqGOJSBKIZ0thLzDa3U8CTgbON7Phh2wzAfjc3QcD9wG/iGMeIVYQ7p6xhNLyChwoLa/g7hlLmPleadTRRCQJxK0oeMzO4GVmsBw69+fFwPTg+XPAGDOzeGUSuGfWcioqqw5aV1FZxT2zlkeUSESSSVz7FMws3czeBzYBhe6+4JBNegPrANx9P7AN6FLHfq43syIzKyorK4tn5BZvQ3nFYa0XkdQS16Lg7lXufjLQBxhmZl89wv1MdfcCdy/o1q3Ru7SlAb1ysg5rvYikloRcfeTu5cA84PxD3ioF+gKYWQbQkViHs8TJxDGDv7SuTUYad44fEkEaEUk2cSsKZtbNzHKC51nAOODjQzZ7HrgqeH4ZMNfdD+13kCa0ZVclAF3btaK68+a0AZ119ZGIAPEdEK8nMN3M0okVnz+5+wtm9hOgyN2fBx4HnjKzEmArcEUc86S8HXsqefTVlYwc0o0nrhkGwF1/+YAZi0prLk8VkdQWt6Lg7h8AQ+tY/8Naz/cAl8crgxzsd2+soXx3JXeMy69Zd8vowfxl0XoemlfCz75xQoTpRCQZ6I7mFLFtdyW/fW0V447rwYl9cmrW9+nUlitO68ef3l3HJ1t2R5hQRJKBikKK+O1rq9ixZ/9BrYRqt4weTHqa8cDcFREkE5FkoqKQArbs3Mu0N1Zz4Yk9+UrPDl96v0eHNlw5vD8zFq1nVdnOOvYgIqlCRSEFPPrqKvZUVjFpbF6929w0chCtM9L51Wy1FkRSmYpCC7dp+x6efGsNl5zcm8Hd29e7Xdd2rbn6rFz+9sEGlm/ckbiAIpJUVBRauF+/spLKKue2MfW3EqpdP2Ig2a0yuK+wOAHJRCQZqSi0YBvKK3h6wSdcfmofcrtmN7p9p+xWTDh7AH9ftpGlpdsSkFBEko2KQgv24NwSHOeW0V8e2qI+E0YMoGNWploLIilKRaGF+mTLbv5ctI7vDOtHn05tQ3+uQ5tMrj9nIHM+3sSiTz6PY0IRSUYqCi3UA3NXkJ5m3DwqfCuh2tVn5tIlu5VaCyIpSEWhBVpZtpMZi9Zz5fD+9OjQ5rA/n906gxvPHcRrKzazYJUGrRVJJSoKLdD9s1fQOiOdm0YOOuJ9XDm8P93bt+bewmI0cK1I6lBRaGGWb9zB3z7YwNVn5dK1Xesj3k9Wq3RuHjWYd1Zv5Y0StRZEUoWKQgtzX2Ex7VplcMM5A496X1cM60uvjm345cvL1VoQSREqCi3I0tJt/H3ZRq49ewA5bVsd9f5aZ6Rz65g83l9Xzrzlm5ogoYgkOxWFFmRKYTEdszKZMGJAk+3zslP70K9zW+59WX0LIqlARaGFWPTJ58z9eBPXnzOQDm0ym2y/melpTByTx7IN25m1bGOT7VdEkpOKQgtxX2ExXbJbcfWZuU2+70uG9mZgt2ymFBZTdUCtBZGWTEWhBViwaguvrdjMTSMHkd266WdYTU8zJo3Np/iznbzwwYYm37+IJA8VhWbO3bn35WK6t2/NlcP7x+04F57Qk2OPac/9s1ewv+pA3I4jItFSUWjmXi/ZzDtrtnLzqMG0yUyP23HS0ozbx+azavMuZr6v1oJIS6Wi0IxVtxJ6dWzDFcP6xv1444/vwVd7d+D+OcVUqrUg0iI1WhTMbJCZtQ6ejzSz28wsJ/7RpDHzlm/i/XXl3Domj9YZ8WslVDMzJo8bwrqtFfy5aH3cjyciiRempfAXoMrMBgNTgb7A03FNJY2qbiX069yWy07tk7DjjhzSjVP65fDg3BXsqaxK2HFFJDHCFIUD7r4f+AbwoLvfCfRs7ENm1tfM5pnZh2a2zMwm1rHNSDPbZmbvB8sPD/9HSE2zlm1k2YbtTByTR2Z64s4CmhmTzxvCp9v28Ow7nyTsuCKSGGF+m1Sa2XeAq4AXgnVh7o7aD0x29+OA4cDNZnZcHdu95u4nB8tPQqVOcVUHnCmFxQzsls0lQ3sn/PhnDurC6QM68/ArK6nYp9aCSEsSpihcA5wB/Je7rzazAcBTjX3I3T9190XB8x3AR0Dif4O1QC98sIHiz3YyaWw+6WmW8ONXtxbKduzlqbfXJPz4IhI/jRYFd//Q3W9z92eC16vd/ReHcxAzywWGAgvqePsMM1tsZi+Z2fH1fP56Mysys6KysrLDOXSLs7/qAPfPXsGxx7TnwhMaPYsXN8MGdGZEXlcemb+KnXv3R5ZDRJpWmKuPzjKzQjMrNrNVZrbazFaFPYCZtSPWWX27u28/5O1FQH93Pwl4EJhZ1z7cfaq7F7h7Qbdu3cIeukX663ulrNq8i0nj8kmLoJVQ2+TzhrB11z6mv7km0hwi0nTCnD56HJgCnA2cBhQEj40ys0xiBeEP7j7j0Pfdfbu77wyevwhkmlnXkNlTzr79B3hg7gpO6N2R847rEXUcTu6bw9ivdOfR+SvZVlEZdRwRaQJhisI2d3/J3Te5+5bqpbEPmZkRKygfufuUerY5JtgOMxsW5NE0X/X488J1rNtawR3j8gm+tshNGpfP9j37efz11VFHEZEmEGb0tHlmdg8wA9hbvbK6E7kBZwHfA5aY2fvBun8D+gWffwS4DLjJzPYDFcAVrkH767SnsoqH5pZwSr8cRg5JnlNox/fqyAUnHMO011dzzZm5dMo++sl9RCQ6YYrC6cFjQa11Doxu6EPu/jrQ4J+z7v4Q8FCIDCnv2Xc+4dNte/jl5SclTSuh2u1j83lp6UYefXUVd33t2KjjiMhRaLQouPuoRASR+lXsq+KheSsZPrAzZw7qEnWcL8nv0Z6LT+rF9DfXMOHsAXRr3zrqSCJyhMJcfdTRzKZUXxJqZveaWcdEhJOYp95ew+ade5l83pCkayVUmzg2n31VB/jNKyujjiIiRyFMR/M0YAfwrWDZDvwunqHkCzv37uc3r6xkRF5XTsvtHHWceg3oms2lQ3vz+wVr2bhtT9RxROQIhSkKg9z9R+6+Klh+DAyMdzCJeeKN1Xy+u5LJ5w2JOkqjbhuTh7vz8LySqKOIyBEKUxQqzOzs6hdmdhaxK4UkzrZVVDL11VWM/Up3Tu6b/KOV9+3clm8V9OXZdz9h/ee7o44jIkcgTFG4CXjYzNaY2VpiVwvdGN9YAvD4a6vYvmc/k8blRx0ltFtGD8bMeHCOWgsizVGYsY/eD4ahOBE4wd2Huvvi+EdLbVt37WPaG2u44IRjOL5X8+nX79kxi++e3o/nFq1nzeZdUccRkcNUb1EwsyuDxzvM7A7gOuC6Wq8ljh59dSW79u3n9rHNp5VQ7aaRg8hMN+6fsyLqKCJymBpqKWQHj+3rWNrFOVdKK9uxlyffXMvFJ/Uiv0f7qOMctu7t23DVmbnMfL+UFZ/tiDqOiByGem9ec/dHg6ez3f2N2u8Fnc0SJ795ZSX7qg4wsRm2EqrdcM4gfv/WWn41ewUPf/eUqOOISEhhOpofDLlOmsCn2yr4/YK1fPOU3gzomt34B5JU5+xWTDh7AP+35FM+3HDoiOkikqzqbSmY2RnAmUC3Q/oQOgDp8Q6Wqh6eV4K7c+vovKijHLUJIwbyxJtrmFJYzGNXFTT+ARGJXEMthVbE+g4yOLg/YTux0U2lia3bups/vruObxX0pW/ntlHHOWodszL5wYiBzP7oMxavK486joiE0FCfwnxgvpk94e5rE5gpZT04dwVmxi2jB0cdpclcc/YApr2xmimFxUy/dljUcUSkEWH6FHab2T1m9qKZza1e4p4sxazevIu/LCrlu6f3o2fHrKjjNJl2rTO48dxBzC8uo2jN1qjjiEgjwhSFPwAfAwOAHwNrgHfjmCkl3T+7mMx046aRg6KO0uS+f0YuXdu15t6Xi6OOIiKNCFMUurj740Clu89392tpZIIdOTwrPtvB/y7ewFVn5tK9fZuo4zS5rFbp3DxqEG+t2sKbJZujjiMiDQhTFKpnZP/UzC40s6FA8o7h3AzdN7uYtpnp3HBOy2slVPvOsH707NiGewuL0YyrIskrTFH4aTCpzmTgn4HHgElxTZVClm3YxotLNjLh7AF0bsHzG7fJTOeW0YNZuPZz5heXRR1HROoRZkC8F9x9m7svdfdR7n6quz+fiHCp4L7CFXRok8GEES1/iorLT+1Ln05ZTFFrQSRphZmOc7qZ5dR63cnMpsU3Vmp4f105sz/6jB+MGEjHrMyo48Rdq4w0bhuTxwfrt1H44WdRxxGROoQ5fXSiu9fceeTunwND4xcpdUwpLKZT20yuOXtA1FES5tKhseE7phQWc+CAWgsiySZMUUgzs07VL8ysMw3c9CbhvLtmK68Wl3HjuYNo1zp1vs6M9DRuH5vHxxt38OLST6OOIyKHCFMU7gXeMrP/NLOfAm8C/9PYh8ysr5nNM7MPzWyZmU2sYxszswfMrMTMPjCzlBlO896Xl9O1XWu+f0Zu1FES7usn9iK/RzvuKyymSq0FkaTS6J+o7v6kmRXxxb0Jl7r7hyH2vR+Y7O6LzKw9sNDMCg/57NeAvGA5HfhN8NikZr5Xyj2zlrOhvIJeOVncOX4Ilwzt3dSHCZ2jtDw2xfWlQ3uR1Sr1xhZMTzMmjc3npj8souCnhZTvroz0v4uIfKGhmdc6BI+dgY3A08GyMVjXIHf/1N0XBc93AB8Bh/6Lvxh40mPeBnLMrOcR/ST1mPleKXfPWEJpeQUOlJZXcPeMJcx8r7QpD3NYOaq9uHRjwnMkiz37qjDg892Vkf53EZGDNdRSeBr4OrAQqN3Gt+B16GsozSyXWOf0gkPe6g2sq/V6fbCuyU423zNrORWVVQetq6is4t/+uoQ3Enh37f8t+fRLOfZUHuCeWctT8q/jXxYWc+iJo4rKqpT9PkSSRUNF4efB41fcfc+RHsDM2gF/AW539yOabcXMrgeuB+jXr99hfXZDrb/Ma9u9ryqhRWH3vqo619eXr6Wr7+dO1e9DJFk0VBTuB04l1rF8RB3AZpZJrCD8wd1n1LFJKdC31us+wbqDuPtUYCpAQUHBYfVM9srJOuiUTbXeOVm8cVfihnA66+dz68zRK6fljIh6OOr775Kq34dIsmjo6qNKM5sK9AmuEDpoaWzHZmbA48BH7j6lns2eB74fXIU0HNjm7k16neKd44eQlXlwZ25WZjp3jh/SlIdpNjmSRV3fR5vMtJT9PkSSRUMtha8DY4HxxPoVDtdZwPeAJWb2frDu34B+AO7+CPAicAFQAuwGrjmC4zSo+vx01FcfJUuOZHHo9+HA+ccfk7Lfh0iysMbGoDGzk9x9cYLyNKqgoMCLioqijiFN7MrHFvDxxu3Mv3MU2Sl0M59IopjZQndvdLL0hi5J/Zfg6XVHcvpI5HDccV4+m3fuY/pba6KOIpLSGvqT7KPgUX+WS9yd0q8To4/tzqPzV3Hl8P50aNPyBwgUSUb1thTc/W/B4/TqBXgK+GvwXKRJ3TEun20VlUx7fXXUUURSVpihs582sw5mlg0sBT40szvjH01SzVd7d2T88T14/LXVlO/eF3UckZQUZkC844Kbzi4BXgIGELuqSKTJTRqXz859+/nta6uijiKSksIUhczgJrRLgOfdvRK+NEKBSJM49pgOfP3EXvzujTVs2bk36jgiKSdMUXgUWANkA6+aWX/giIarEAnj9rF57Kms4pH5K6OOIpJywszR/IC793b3C4LRTNcCoxKQTVLUoG7t+MbQPjz51lo2bT/iYbdE5AiE6WieGHQ0m5k9bmaL+GJuBZG4mDgmj6oDzsPzSqKOIpJSwpw+ujboaD4P6ESsk/nnDX9E5Oj069KWywv68sw76+ocOE9E4iNMUbDg8QLgKXdfVmudSNzcOnowAA/NXRFxEpHUEaYoLDSzl4kVhVnB1JoH4htLJDaM9neG9eXPRev5ZMvuqOOIpIQwRWECcBdwmrvvBloRh9FMRepy86jBpKcZ989Ra0EkEcJcfXQAWA3km9k5wPFATryDiQB079CG75/Rn7++t56STTujjiPS4oW5+ug64FVgFvDj4PE/4htL5As3njuINpnpai2IJECY00cTgdOAte4+ChgKlMc1lUgtXdq15pqzcvnb4g18vFH3TYrEU5iisMfd9wCYWWt3/xjQnImSUD8YMZD2rTO4r7A46igiLVqYorDezHKAmUChmf0vsDa+sUQOltO2FdeNGMisZZ+xZP22qOOItFhhOpq/4e7l7v4fwL8DjxMbHE8koa49O5ectplMKVwedRSRFquh6Tg7H7oAS4DXgXYJSygSaN8mk+vPGci85WUsXPt51HFEWqSGWgoLiU3FubDWUlTrUSThrj4zl67tWqm1IBIn9c7R7O4DEhlEJIy2rTK48dxB/PT/PuLtVVsYPrBL1JFEWpQw9yl8w8w61nqdY2bqU5DIXDm8Pz06tGbKy8W4a74nkaYU5uqjH7l7zeUe7l4O/Ch+kUQa1iYznVtGDeadNVt5bcXmqOOItChhikJd29R72qmamU0zs01mtrSe90ea2TYzez9YfhgiiwgA3zqtL71zsri3UK0FkaYUpigUmdkUMxsULPcR62xuzBPA+Y1s85q7nxwsPwmxTxEAWmekc9uYwSxeV86cjzZFHUekxQhTFG4F9gF/DJY9wM2NfcjdXwW2HlU6kQZcekof+ndpy5TCYg4cUGtBpCmEuXltl7vf5e4FwOnAf7v7riY6/hlmttjMXjKz45ton5IiMtPTuH1sHh9+up1ZyzZGHUekRQhz9dHTwRzN2cRuXvvQzO5sgmMvAvq7+0nAg8SG0agvw/VmVmRmRWVlZU1waGkpLjqpN4O7t2NKYTFVai2IHLUwp4+OC+ZovgR4CRhAbJ7mo+Lu2919Z/D8RSDTzLrWs+1Udy9w94Ju3bod7aGlBUlPM24fm8eKTTt54YMNUccRafbCFIVMM8skVhSed/dK4Kj/JDOzY8zMgufDgixbjna/knou+GpPjj2mPb+avYL9VZopVuRohCkKjwJrgGzgVTPrDzQ6qL2ZPQO8BQwxs/VmNsHMbjSzG4NNLgOWmtli4AHgCte1hXIE0tKMO8bls3rzLma8Vxp1HJFmzY7k97CZZbj7/jjkaVRBQYEXFWnoJTmYu3Pxw2+wddc+5k4eSauMMH/viKQOM1sYXDDUoIZGSb0yeLzj0AW4rQmzihw1s1hrYf3nFfypaF3UcUSarYb+nMoOHtvXs4gklXPzu3Fq/048NLeEPZVVUccRaZYaGiX10eDxx4mLI3LkzIzJ5+Xzj79dwNMLPuHaszXQr8jhCjOG0QBidzXn1t7e3S+KXyyRI3PmoK6cMbALv35lJd8Z1o+sVulRRxJpVsL0xs0kdvXRg8C9tRaRpDT5vHw279zLk2+tiTqKSLPTaEsB2OPuD8Q9iUgTKcjtzLn53Xhk/kq+O7w/7VqH+d9cRCBcS+F+M/uRmZ1hZqdUL3FPJnIU7hiXz+e7K/nd66ujjiLSrIT5E+oEYsNajAaqbxf14LVIUjqpbw7jjuvB1NdW8f0zcunYNjPqSCLNQpiWwuXAQHc/191HBYsKgiS9O8bls2PPfh57fVXUUUSajTBFYSmQE+8gIk3tKz07cOEJPZn2+mq27toXdRyRZiFMUcgBPjazWWb2fPUS72AiTWHSuDwqKqt4dP7KqKOINAth+hR+FPcUInEyuHt7Lj65N9PfWsOEEQPo3r5N1JFEklqYmdfm17UkIpxIU5g4Jo/KKufX89RaEGmMhpKUFi+3azaXndKHpxd8wqfbKqKOI5LUVBQkJdw6ZjCO89DckqijiCS1hobOnhM8/iJxcUTio0+ntlxxWj/++O461m3dHXUckaTVUEuhp5mdCVxkZkNr382sO5qlObp51GDS0owH5qyIOopI0mro6qMfAv8O9AGmHPKe7miWZueYjm248vT+PPHmam4aOYiB3dpFHUkk6dTbUnD359z9a8D/1LqTWXc0S7N208hBtM5I5361FkTqFOaS1P80s4vM7JfB8vVEBBOJh27tW3PVmbk8v3gDxZ/tiDqOSNJptCiY2X8DE4EPg2Wimf0s3sFE4uWGcwaS3SqD+wqLo44iknTCXJJ6ITDO3ae5+zTgfECtBWm2OmW34tqzB/DS0o0sLd0WdRyRpBL2PoXaA+J1jEcQkUSacPYAOrRRa0HkUGGKwn8D75nZE2Y2HVgI/Fd8Y4nEV8esTG44dxBzPt7Ee598HnUckaQRpqP5GWA4MAP4C3CGu/+xsc+Z2TQz22RmS+t538zsATMrMbMPdO+DJNrVZ+bSObsVU9RaEKkRavJad/8UONzhsp8AHgKerOf9rwF5wXI68JvgUSQhsltncOO5A/nZix9T8NNCtuzcR6+cLO4cP4RLhvaOOp5IJOI29pG7vwpsbWCTi4EnPeZtIMfMesYrj0hdcrJaAbB55z4cKC2v4O4ZS5j5Xmm0wUQiEuWAeL2BdbVerw/WiSRMXTexVVRWcc+s5RGkEYleg0XBzNLN7ONEhWkgx/VmVmRmRWVlZVHHkRZkQ3ndQ2nXt16kpWuwKLh7FbDczPrF4dilQN9ar/sE6+rKMdXdC9y9oFu3bnGIIqmqV07WYa0XaenCnD7qBCwzszlNPEfz88D3g6uQhgPbgg5tkYS5c/wQsjLTD1qXlZnGneOHRJRIJFphrj769yPZsZk9A4wEuprZemJzPWcCuPsjwIvABUAJsBu45kiOI3I0qq8yumfWckqDU0ZXDOunq48kZZm7N76RWX8gz91nm1lbIN3dIxlNrKCgwIuKiqI4tLRw+6sOcN59r5KZnsZLE0eQlmZRRxJpMma20N0LGtsuzIB4PwCeAx4NVvUGZh5dPJHkk5Gexu3j8ln+2Q5eWKIzmZKawvQp3AycBWwHcPcVQPd4hhKJytdP6MmQHu351exi9lcdiDqOSMKFKQp73X1f9QszyyA285pIi5OWZkwal8eqsl3MfH9D1HFEEi5MUZhvZv8GZJnZOODPwN/iG0skOuOPP4bje3Xg/jnFVKq1ICkmTFG4CygDlgA3ELtq6P/FM5RIlMyMyefls25rBX8uWh91HJGEavSSVHc/EAyZvYDYaaPlHuaSJZFmbNSQ7gztl8NDc1fwzVN70zojvfEPibQAYa4+uhBYCTxAbNTTEjP7WryDiUTJzJg8bggbtu3h2XfWNf4BkRYizOmje4FR7j7S3c8FRgH3xTeWSPTOGtyFYQM689C8Eir2VUUdRyQhwhSFHe5eUuv1KiCSG9dEEinWWsinbMdefv/22qjjiCREvX0KZnZp8LTIzF4E/kSsT+Fy4N0EZBOJ3OkDuzAiryu/mb+S75zej3atQ81LJdJsNdRS+IdgaQN8BpxLbCyjMkBDSErKuGNcPlt37WP6m2uijiISd/X+2ePuGqBOBBjarxNjju3Oo/NXcuXw/nTMyow6kkjchLn6aICZTTGzGU08dLZIszFpXD7b9+zn8ddXRx1FJK7CnCCdCTxO7C5m3d4pKemrvTvyta8ew7TXV3PNmbl0ym4VdSSRuAhz9dEed3/A3ee5+/zqJe7JRJLMpHH57Nq3n6mvrYo6ikjchCkK95vZj8zsDDM7pXqJezKRJJPfoz0XndSLJ95YQ9mOvVHHEYmLMEXhBOAHwM+J3ch2L/DLeIYSSVYTx+Sxd38Vj8xfGXUUkbgI06dwOTCw9vDZIqlqYLd2XHpKH556ey0/GDGQYzq2iTqSSJMK01JYCuTEO4hIczFxTB4HDjgPzytpfGORZiZMUcgBPjazWbokVQT6dm7Lt07ry7PvfsL6z3dHHUekSYU5ffSjuKcQaWZuHT2Y5xau58E5JfzishOjjiPSZMLMp6DLT0UO0bNjFv84rB9Pvb2Wm0YOIrdrdtSRRJpEmDuad5jZ9mDZY2ZVZrY9EeFEktk/jRpEZrpx/5wVUUcRaTKNFgV3b+/uHdy9A7GB8L4J/DruyUSSXPf2bbjqjFxmvl9KySaNJi8tQ5iO5hoeMxMYH2Z7MzvfzJabWYmZ3VXH+1ebWZmZvR8s1x1OHpGo3XDuINpmpnPfbLUWpGVotE+h1rwKECsiBcCeEJ9LBx4GxgHrgXfN7Hl3//CQTf/o7reEjyySPDpnt+Laswfw4NwSbh65neN6dYg6kshRCdNS+Iday3his65dHOJzw4ASd18V3Pj2bMjPiTQr1509kPZtMrhvdnHUUUSOWpirj450XoXeQO0Zz9cDp9ex3TfN7BygGJjk7polXZqVjm0z+cGIgUwpLGbxunJO6qt7PaX5amg6zh828Dl39/9sguP/DXjG3fea2Q3AdGB0HVmuB64H6NevXxMcVqRpXXNWLr97YzVTCouZfu2wqOOIHLGGTh/tqmMBmAD8a4h9lwJ9a73uE6yr4e5b3L16uMnHgFPr2pG7T3X3Ancv6NatW4hDiyRW+zaZ3HDuIOYXl1G0ZmvUcUSOWL1Fwd3vrV6AqcQuR72GWN/AwBD7fhfIC2ZuawVcARw0PIaZ9az18iLgo8PML5I0vn9Gf7q2a829L6tvQZqvBjuazayzmf0U+IDYqaZT3P1f3X1TYzt29/11f01qAAAJyUlEQVTALcAsYr/s/+Tuy8zsJ2Z2UbDZbWa2zMwWA7cBVx/FzyISqbatMvinkYN4a9UW3izZHHUckSNi7l73G2b3AJcSayU87O47ExmsPgUFBV5UVBR1DJE67amsYuQ9r9C7UxbP3XgGZhZ1JBEAzGyhuxc0tl1DLYXJQC/g/wEbag11sUPDXIjUrU1mOreMHszCtZ8zv7gs6jgih62hPoU0d8+qPcxFsLQPhrwQkTp8q6AvfTplMaWwmPpa4iLJ6rCGuRCRxrXKSOO2MXl8sH4bhR9+FnUckcOioiASB5cO7c2ArtlMKSzmwAG1FqT5UFEQiYOM9DQmjsnj4407eGnpxqjjiISmoiASJ/9wUi/yurfjvtnFVKm1IM2EioJInKSnGZPG5VOyaSfPLy5t/AMiSUBFQSSOzj/+GI7r2YFfzV5BZdWBqOOINEpFQSSO0tKMO8bls3bLbmYsWh91HJFGqSiIxNmYr3TnpL45PDCnhL37q6KOI9IgFQWRODMzJo/Lp7S8gj+9q+lCJLmpKIgkwIi8rpyW24mH5pWwp1KtBUleKgoiCWBmTD5vCJ9t38vv314bdRyReqkoiCTI8IFdOGtwFx6Zv5Ld+/ZHHUekTioKIgl0x7ghbN65j+lvqrUgyUlFQSSBTu3fiVFDuvHoqyvZsacy6jgiX6KiIJJgd4wbQvnuSqa9vibqKCJfoqIgkmAn9OnI+ON78NhrqyjfvS/qOCIHUVEQicCkcfns3Lef3762KuooIgdRURCJwLHHdODCE3ryuzfWsGXn3qjjiNRQURCJyO1j89lTWcUj81dGHUWkhoqCSEQGd2/HJUN78+Rba9m0fU/UcUQAFQWRSE0ck0fVAefXr6i1IMlBRUEkQv27ZHN5QR+eXvAJpeUVUccRiW9RMLPzzWy5mZWY2V11vN/azP4YvL/AzHLjmUckGd0yOg+Ah+aWRJxEBDLitWMzSwceBsYB64F3zex5d/+w1mYTgM/dfbCZXQH8Avh2vDKJJKPeOVl8Z1hfnnxrLXM//oxN2/fSKyeLO8cP4ZKhvROaZeZ7pdwzazkbyisiy6Ac0eaIW1EAhgEl7r4KwMyeBS4GaheFi4H/CJ4/BzxkZubumuVcUsqgbtk48Nn22OWppeUV3D1jCUDCfgnNfK+Uu2csoSIY2juKDMoRfQ6L1+9fM7sMON/drwtefw843d1vqbXN0mCb9cHrlcE2m+vbb0FBgRcVFcUls0hUzvr53Dr7FDLSjAFdsxOSYfXmXew/8OXfB4nMoBzhc/TOyeKNu0aH3o+ZLXT3gsa2i2dLocmY2fXA9QD9+vWLOI1I09tQTyfz/gNOXo92CcmwYtPOyDMoR/gc9f0/c7TiWRRKgb61XvcJ1tW1zXozywA6AlsO3ZG7TwWmQqylEJe0IhHqlZNVZ0uhd04Wv/7uqQnJUF9rJZEZlCN8jl45WXE5XjyvPnoXyDOzAWbWCrgCeP6QbZ4HrgqeXwbMVX+CpKI7xw8hKzP9oHVZmencOX5ISmVQjuhzxK2l4O77zewWYBaQDkxz92Vm9hOgyN2fBx4HnjKzEmArscIhknKqOwyjvNIlGTIoR/Q54tbRHC/qaBYROXxhO5p1R7OIiNRQURARkRoqCiIiUkNFQUREaqgoiIhIjWZ39ZGZlQFro85xlLoC9Q7lkYL0fRxM38cX9F0c7Gi+j/7u3q2xjZpdUWgJzKwozKVhqULfx8H0fXxB38XBEvF96PSRiIjUUFEQEZEaKgrRmBp1gCSj7+Ng+j6+oO/iYHH/PtSnICIiNdRSEBGRGioKCWRmfc1snpl9aGbLzGxi1JmiZmbpZvaemb0QdZaomVmOmT1nZh+b2UdmdkbUmaJkZpOCfydLzewZM2sTdaZEMrNpZrYpmKGyel1nMys0sxXBY6emPq6KQmLtBya7+3HAcOBmMzsu4kxRmwh8FHWIJHE/8Hd3PxY4iRT+XsysN3AbUODuXyU2/H6qDa3/BHD+IevuAua4ex4wJ3jdpFQUEsjdP3X3RcHzHcT+0Sd2cPYkYmZ9gAuBx6LOEjUz6wicQ2yOEdx9n7uXR5sqchlAVjArY1tgQ8R5EsrdXyU2z0xtFwPTg+fTgUua+rgqChExs1xgKLAg2iSR+hXwL8CBqIMkgQFAGfC74HTaY2aWuNnhk4y7lwK/BD4BPgW2ufvL0aZKCj3c/dPg+UagR1MfQEUhAmbWDvgLcLu7b486TxTM7OvAJndfGHWWJJEBnAL8xt2HAruIw6mB5iI4V34xsWLZC8g2syujTZVcgqmLm/zyURWFBDOzTGIF4Q/uPiPqPBE6C7jIzNYAzwKjzez30UaK1HpgvbtXtxyfI1YkUtVYYLW7l7l7JTADODPiTMngMzPrCRA8bmrqA6goJJCZGbFzxh+5+5So80TJ3e929z7unkusA3Guu6fsX4LuvhFYZ2bVs7GPAT6MMFLUPgGGm1nb4N/NGFK4472W54GrgudXAf/b1AdQUUiss4DvEfur+P1guSDqUJI0bgX+YGYfACcDP4s4T2SCFtNzwCJgCbHfVSl1d7OZPQO8BQwxs/VmNgH4OTDOzFYQa039vMmPqzuaRUSkmloKIiJSQ0VBRERqqCiIiEgNFQUREamhoiAiIjVUFCSlmVlVrcuD3zezJruL2Mxya49wKdIcZEQdQCRiFe5+ctQhRJKFWgoidTCzNWb2P2a2xMzeMbPBwfpcM5trZh+Y2Rwz6xes72FmfzWzxcFSPSRDupn9NpgX4GUzywq2H2RmfzezhWb2mpkdG6y/PJg/YLGZvRrJDy8pTUVBUl3WIaePvl3rvW3ufgLwELERXQEeBKa7+4nAH4AHgvUPAPPd/SRiYxYtC9bnAQ+7+/FAOfDNYP1U4FZ3PxX4Z+DXwfofAuOD/VzU1D+sSGN0R7OkNDPb6e7t6li/Bhjt7quCQQw3unsXM9sM9HT3ymD9p+7e1czKgD7uvrfWPnKBwmBCFMzsX4FMYgWmDFhe65Ct3f0rZvYIMAj4EzDD3bfE4ccWqZf6FETq5/U8Pxx7az2vArKItdDL6+rLcPcbzex0YpMPLTSzU1UYJJF0+kikft+u9fhW8PxNvpgW8rvAa8HzOcBNUDPvdMf6dhrMobHazC4PtjczOyl4PsjdF7j7D4m1Jvo24c8j0igVBUl1h/Yp1B51slMwYulEYFKw7lbgmmD994L3CB5HmdkSYCHQ2Nzb3wUmmNliYv0PFwfr7wk6t5cSK0CLj/YHFDkc6lMQqUPQp1Dg7pujziKSSGopiIhIDbUURESkhloKIiJSQ0VBRERqqCiIiEgNFQUREamhoiAiIjVUFEREpMb/B59nKVf0Vla6AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f51508f0110>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(range(1, len(ppn.errors_) + 1), ppn.errors_, marker='o')\n",
    "plt.xlabel('Epoches')\n",
    "plt.ylabel('Number of misclassifications')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "通过上图我们可以发现，第6次迭代时，感知机算法已经收敛了，対训练集的预测准确率是100%。接下来我们将分界线画出来："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "from matplotlib.colors import ListedColormap"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_decision_region(X, y, classifier, resolution=0.02):\n",
    "    # setup marker generator and color map\n",
    "    markers = ('s', 'x', 'o', '^', 'v')\n",
    "    colors = ('red', 'blue', 'lightgreen', 'gray', 'cyan')\n",
    "    cmap = ListedColormap(colors[:len(np.unique(y))])\n",
    "    \n",
    "    # plot the decision surface\n",
    "    x1_min, x1_max = X[:, 0].min() - 1, X[:, 0].max() + 1\n",
    "    x2_min, x2_max = X[:, 1].min() - 1, X[:, 1].max() + 1\n",
    "    \n",
    "    xx1, xx2 = np.meshgrid(np.arange(x1_min, x1_max, resolution),\n",
    "                          np.arange(x2_min, x2_max, resolution))\n",
    "    Z = classifier.predict(np.array([xx1.ravel(), xx2.ravel()]).T)\n",
    "    Z = Z.reshape(xx1.shape)\n",
    "    plt.contourf(xx1, xx2, Z, alpha=0.4, cmap=cmap)\n",
    "    plt.xlim(xx1.min(), xx1.max())\n",
    "    plt.ylim(xx2.min(), xx2.max())\n",
    "    \n",
    "    # plot class samples\n",
    "    for idx, cl in enumerate(np.unique(y)):\n",
    "        plt.scatter(x=X[y == cl, 0], y=X[y==cl, 1],\n",
    "                   alpha=0.8, c=cmap(idx),\n",
    "                   marker=markers[idx], label=cl)\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEKCAYAAAARnO4WAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3X+UHXWZ5/H30wmmiWnIBBJGJ0Ark8SNoPxIohgIQoQFZfwxyhncMzsHyJ4GdVlc1nENnD3OOA7tj8XRZTCYpWFkYWBExeEwgPwSG6IDnSAQQJJgAA0CieEQEkMnhn72j6rqvvf2vXXr/qj7o+rzOqdPd9W9VfdbfeDpylPP9/mauyMiItnX0+4BiIhIayjgi4jkhAK+iEhOKOCLiOSEAr6ISE4o4IuI5IQCvohITijgi4jkhAK+iEhOTE3z5GY2E7gaOBJw4Dx3/3ml98+YcbAfdFB/mkMSEcmUX/963e/cfXaS96Ya8IFvAXe6+yfM7E3A9Lg3H3RQP5deujblIYmIZMf559vzSd+bWsA3swOBZcA5AO6+F9ib1ueJiEi8NHP4bwO2Adea2S/M7Goze3Ppm8xswMzWmtnaXbu2pTgcEZF8SzPgTwWOBVa5+zHA74EvlL7J3Ve7+yJ3XzRjRqI0lIiI1CHNHP4WYIu7PxRuf58yAb+aqVP/wNvfvoXp00ebOrg07d7dy+bNc9m3b792D0VEZFxqAd/dXzKz35jZAnffACwHnqr1PG9/+xYOPbSPvr5+zKz5A20yd2fnzu3AFjZufFu7hyMiMi7tKp0LgRvCCp3NwLm1nmD69NGuCfYAZkZf30FMn67nESLSWVIN+O7+KLCo0fN0S7CPdNt4RSQfNNNWRCQnFPBrsGnT05xxxvHMnTuNK6/83+0ejohITdLO4WfKzJmzuOyy/8Mdd/yo3UMREalZpgL+zNMW0/O7rZP2jx08h1fvGmn4/LNnz2H27Dncffe/NXwuEZFWy1TA7/ndVsYOmjx5q9wfARGRvFEOX0QkJxTwqxgaupKTTz6ak08+mpde+m27hyMiUrdMpXTSsGLFZ1ix4jPtHoaISMMU8Gvw8ssvcdppi9i58zV6enpYvfqbPPjgU/T1HdDuoYmIVJWpgD928JyKVTrNcMghf8xjj21pyrlERFotUwG/GaWXIiJZpYe2IiI5oYAvIpITCvgiIjmhgC8ikhMK+CIiOaGAn8BFF53HwoVzWLbsyHYPRUSkbpkL+O7x2/U4++xzuOmmOxs/kYhIG2Uq4F93HaxaNRHk3YPt665r7LzHH7+MmTNnNT5AEZE2ykzAd4ddu+CWWyaC/qpVwfauXc250xcR6WaZmWlrBp/6VPDzLbcEXwAf+1iwX+uKi0jeZeYOH4qDfkTBXkQkkKmAH6VxChXm9EU6VRrFBiKlMhPwC3P2H/sY3HVX8L0wp1+v88//JB/84PE888wG3v3uudxww1DzBi65NzwM99xTXGxwzz3BfpFmylQOf8aM4px9lN6ZMaOxtM53vnNjcwYpUsId9uyBhx8Otj/wgSDYP/wwLFkSvK6UpDRLZgI+wF/9VfH/IFHQ1/8w0qnMgiAPQZCPAv+SJcF+/bcrzZRqSsfMnjOz9Wb2qJmtTfOzJj4zfluk0xQG/Ui3Bns9i+hsrcjhn+zuR7v7onpP4F32X023jVfaK8rZFyrM6XcLPYvofB3/0Hb37l527tzeNUHU3dm5czu7d/e2eyjSBaKgGOXsL7kk+P7ww90V9AufRUTjjq5rz57uuY6sSzuH78BdZubAd9x9da0n2Lx5LrCF6dO3NX1wadm9uzcct0g8M5g2rThnH6V3pk3rnrSOnkV0h7QD/gnu/oKZzQHuNrOn3b3oH3hmNgAMAMyaddikE+zbtx8bN74t5WGKtM+yZZOLDZoRJEsrfNKu+InGHQV7SH4drR5rXqWa0nH3F8LvW4FbgCVl3rPa3Re5+6IZM2anORyRjtXsYoN25NPrfRah3H/rpBbwzezNZtYX/QycBjyR1ueJSKAd+fR6n0Uo999aaaZ0DgFuseBWZSrwz+6upvKSW61KW7Qjn57kWUSl61fuv3VSC/juvhl4d1rnF+kmw8PBHWsUxKI72WnTghx+szWST69X3LOIatff6rHmVceXZYp0u3amWAq1osyz3LOIatc/NpaNeQjdIFOtFUQ6UavTFqX59ML+PND6u+e461++HO69t3PGmnW6wxdpgUbaJ4yNxW+X+6xp02Dx4uJ8+uLF6df2V2qtUOn6e3rK5/6XLCnO/cd9hiSngC/SAvWmWIaGgvbeUZAfGwu2hzqwQ3dceWXc9Uc5/NLc/7JlKtlsNgV8kZTVW7I4Ngajo/D00xNBf9WqYHt0tPKdfpQzHxkpzpmPjKT7zKBSnn50FO6+O/7668n9606/dsrhi6Ss3vYJPT1Be+9vfzsI8hddFOxfsCDY31Phdq3RZwbVykfrKa984IHar18lm82nO3yRFohLW8R58EGYP784pTF/frA/Tr3PDKqlUOJej/vMeq8/S62jO4ECvkiL1No+wT1Ih9x+e5C+cQ++3357sD8upVHPM4Mk5ZONlFfW0z4iK62jO4VSOiIdyh02boS9e+FNb4LZs2HbtmB748bKd7r1lmUmSaG0sryy08pLs0ABXzKr3lYGccel1R5hbKw4Jx9t778/LFwIO3YE+2fPhgMPDPb39FQeT71tDqrN0I17vdltnrPSOrqTKOBLJtXbyiDuOEinPcLQUJCiiR7ERtU4vb1w3nlBhcvIyMT758+HU0+tfo31tDk48cTyKZTS95Z7PY02z2m1js4r5fAlc+ot56tWWjg62vwSwbjSy9dfnwj2heWMIyPB/mrjqbXUsVr5ZJSjr7W8slFpnDOvdIcvmZMkF11PaWF07maWCEall1GQj0ov3/GOYP+DD1ZOaZx4Yvx4ml0+Gc2KLZ3B6149VSSdQXf4kklx5Xz1lhamVSIYBf1CUXonrpwxjWust3wSNCu2GyjgSyZVyjU3UlqYVolglMYpVNhOoVJKI41rrPZ5lWbwppXykuZSSkcyp1o53/LlwfdaSgujgBXl05tVIliYs4/SONH2qlWVZ9SmcY3VrqMdKS9pLt3hS+ZUKueLOjD29NTeubG3N/iK6+pYj56e4LxRsI/SO+94R7A/rn1CXEfMatdYbyfNdqS8pHl0hy+ZFFfO10hpYRolgitWFNfhR0G/UrBPIu4aG1HtvHElndJ+CviSWZXKEmudvVm4nVaJYGlwrxbsC/Pp0R+eKJ++eHFxOWel1FTpcUuWxFfVxP3u0kp5SXMp4EuuZGX2ZiPlldXKOeM+M+53V3qebvy9Zp0CvuROLWmb0u1KLRCSaHaNelybg2rXGNc+IU47Ul7SPHpoK7lUKTUTV0veyOpTadSoVysTrbWcM2npZDtSXtIcCvgiobi2A6+/HnzVuvpUtfPWW6Ne7ypa9R4n2aCUjkiosFVAYX578eKgWdmpp8avPlWtAyU0r0a93mcRWXmGIfWpGPDN7NYEx7/i7uc0bzgi7fXAA/H7588PAn5U8VO4+lRc58q4fHu96u0kqQ6U+RV3h/8fgP8S87oBVzZ3OCLtE60wde+9QfDr64OdO4PtU04J3nPHHRMPaqPVp844I3h/1MK4sCQxKnWEdGrU682ZK9eeT3EB/1J3/2ncwWb2t00ej0jbRXfo0Ve0vWlT+dWnNm0K0jqVSh1BKzdJZ6j40Nbdv1ft4CTvMbMpZvYLM7ut1sGJpKX0IevYWBB0e3uDu/m+vom7/FNOgenTg6+FC4NgD8H3hQuD1aemTJnoXxNZvnwihx/X6iH6g1KocDvuNZFaVH1oa2aLgEuBw8P3G+Du/q6En3ER8EvggHoHKdJMSVaYKsxvQzBZ6cQT41efWr+++F8Eq1bBUUcFOfN6V6CCdFbZknxKUqVzA/DXwHogpvhsMjObC3wI+Hvg4ppHJ9JkpStMFXanXLAA7roL1q4tn3qByq0DNm6EDRsmd7wEOOGE4A9LpVYPUckmFJ938eKJzyx9rVobBJFykgT8be6epGKnnG8Cnwf66jxecq7Zs1OTrDBVbkWn3t7gfZXKGY86Kvj51VdhcDD4ecECOPLI+Jm4ajksrZRk4tUXzexqM/ukmf159FXtIDM7E9jq7uuqvG/AzNaa2dpdu7YlHbfkQForKMWtMBWn2upT8+YVv3/evGRBWS2HpVWSBPxzgaOB04E/C7/OTHDcUuDDZvYccBNwipldX/omd1/t7ovcfdGMGbMTD1yyLY3ZqZFKK0y98UblFZ2qLQw+Ogr33ReUcULw/b77gv3VxhrX6iCtVbYkn5KkdBa7+4JaT+zuK4GVAGb2fuBz7v6XtZ5H8imN2akQv8LUVVfBBRfU/5nlyjmrUcthaaUkd/g/M7OFqY9EpEQa6YxohanClgif+lSw3dsblFfW+plx5Zy9vfW1HE5zlS3JryR3+O8FHjWzZ4E91F6WibvfD9xfzwAlv+JWV2ok2M2bFzRCK8zFz5sX1NPX+5lRyWa5cs5q1HJYWiVJwD899VGIlKhnZaqk592zJyi9jNZ9veeeYDtupai4zyzM9ZcelzRAq+WwtEKSgP8W4El33wlgZgcQ9Nl5Ps2BSb6l1dWxWkfMuJWi1IFSul2SgL8KOLZge1eZfSJNl1ZXx6jzZeGs2Gi/OlBKliV5aGvuE0Vg7j6G+uhLizQ7nVFaQuk+uYRSHSglq5IE/M1m9t/MbL/w6yJgc9oDE0lT4eIk5ZqXiWRRkoB/AfA+4AVgC/AeYCDNQYmkJSqhXL48KJ2E4Pvy5RMllOpOKVlVNTXj7luBs1swFpGWiEooy+2P61yp7pTS7Sre4ZtZ1bv4JO8R6SSlJZTRIt4jI8EfgdHRdNo5iHSCuDv8L5jZ72JeN4Je96ubOySR9FQroTzxRHWnlOyKC/g/JWiUFqfMP4xFJmt2m+NGVCuhjFtsvJOuQ6RWFQO+u5/byoFIdnViXrxSCWVca4UHHui86xCpherpJVVxKzp12qpNSTtXQmdfh0glCviSqrTaHKdB+X3JOgV8SV0UOCvlxTtJI/l9kU5XNeCb2TTg40B/4fvd/UvpDUuyJK02x2mpJ7/fidchUirJHf6/AjuAdQT98EUSS6vNcatl5Tok35IE/Lnurp74UlWlksVp04L2w4V5cffuah2sFsiSBUkC/s/M7Ch3X5/6aKRrxZVeZoVaIEu3i2utsN7MHgdOAB4xsw1m9njBfhGguPSytCXB6GjwNTJS/NrISHe2K1ALZOlmcXf4Z7ZsFNLVqpVeRu9ROaNIe1W8w3f35939eeDL0c+F+1o3ROkGhUE/EgX0uNdEpHWS9MN/Z+GGmU0BjktnONKtKpUsuse/JiKtUzGlY2YrgUuA/c3stWg3sBd1yJQCSVsSqJxRpLmGh2t7f1zztEFg0MwG3X1lg+OSDKtWsggqZxRplsFB4De/ruvYJGWZN5vZsSX7dgDPu/u+uj5VMqdayaLKGUWSGR6GTZsmtreumxzcnz3uLFixAgA7P/m5kwT8bwPHAo8TpHSOAp4ADjSzT7n7Xck/TrIsrmRR5YwilQ1e8hqMvg6jQTODpb3rxl976NCLYWVpkmVFXZ+TJOD/Fljh7k8CmNlC4EvA54EfAgr4IiIJVUrJPHvcWcEPK0qDefMy6kkC/vwo2AO4+1Nm9g5332wxt2lm1gsMA9PCz/m+u3+x0QFL9mgVKcmiwcHwh0rBfVJgr++uvRZJAv6TZrYKuCnc/gvgqbCL5h9ijtsDnOLuu8xsP+BBM7vD3f+9sSFLlnTialgitRq85DV49dWifXOmbGdF380MHHpj01IyjUoS8M8BPg18NtxeA3yOINifXOkgd3dgV7i5X/ilymsZ102rYYkADA3B1q3UcNfeTzNTMo2qGvDd/XXg8vCr1K4y+8aFk7TWAX8KXOnuD9UzSMmmbloNS/Jn8OKXJzb+sA/eeIOlveuYB1x/3FfakpJpVJIFUJYCfwMcTvECKG+vdqy7vwEcbWYzgVvM7Eh3f6Lk/APAAMCsWYfVNHjpft20GpZkV6WUzENHXzCxoyjAd35wLydJSmcI+O8Ed+pv1PMh7v6qmf0EOJ2gpLPwtdWEM3cPP3yRUj45o1WkpJXGH6TCpLTMs2dfUubBUXcG9kqSBPwd7n5HrSc2s9nAH8Jgvz9wKvDVWs8j2aVVpCRNgxe/PJ6KiSztXcey3pFgYyYMXNZfcET2qwSSBPyfmNnXCWrux5c4dPdHqhz3FuC7YR6/B/ieu99W90glc7SKlDQqblbqnCnbeeisy8vctfe3ZGydKEnAf0/4fVHBPgdOiTvI3R8HjqlzXJITWkVKalH0ILXMrNQVM0dyd9deiyRVOhVLL0WaQW0XpJzBi18eD+qRpb3ruP6dX5nYUbYMUipJUqVzCHAZ8FZ3PyNsrXC8uw+lPjoRybThYVizpmDHb0pSMmeXS8lk60FqKyVJ6fwTcC1wabi9EfgXguodEZHEBi98YWIjfJi6cuZV47smz0pVSqaZkgT8g939e+GCKLj7PjOrqzxTRLKv8EFq6azUpb3ruP6jPwg2xu/c+wuO7pxZqVmUJOD/3swOImyLYGbvJeiHLyI5NzwMa35U+UHqPOD6Se19ddfeLkkC/sXArcARZrYGmA18ItVRiUjHKp2VunLmVQwcce/EG1Js7yuNSVKl84iZnQQsIFgAZYO7x3XJFJEuNt4gLFK1UVg/epDaHeIWMf/zCi/NNzPc/YcpjUlEWqx0Vmr8g1RQgO9OcXf4fxbzmhPMvBWRLlD0IPVXkxuFQdhLBsKHqf0FryglkxUVA767n9vKgYhIc43PSi33ILVse189TM26JA9tRaTDlZuVWtTetwt7t0vzKeCLdInxWak7Y1IympUqMRTwRTpU0axUGF9xaVnvCAPH3auUjNSsniodAFXpiDTB8HDwfc2PKqRkzipYWbRoZqru3KV2qtIRaZG4WalLgevP/kGZlIzu2qV5VKUjkpLBC18oWm0JqrX3VXCXdCXK4ZvZh4B3Ar3RPnf/UlqDEukWcbNS50zZzkPfHilzlNIx0h5J+uFfBUwHTgauJuij83DK4xLpOMPDsObmWtv7inSOJHf473P3d5nZ4+7+t2Z2OVDzouYi3SJ6kAphGWSl9r6gWanSVZIE/NfD77vN7K3AdoIFykW63tAQbH1y8oPUOVO2B9+Bh467QLl2yYQkAf82M5sJfB14hKBC5+pURyWSotrb+yrnLtmQJOB/zd33AD8ws9sIHtyOpjsskfoNDpbsKNfet2hWaj8K6pIHSQL+z4FjAcLAv8fMHon2iXSC2Pa+mpUqAsTPtP1j4E+A/c3sGILFTwAOIKjaEWmZogepd07uJVM0K3XSg1TdvYtA/B3+fwTOAeYC3yjY/xpwSYpjkpwbHCRoEBYJg3vRg9SzL9esVJEaxc20/S7wXTP7uLv/oNL7RJqhtL3vs4eeBHPmTLxBKRmRhiXJ4a8xsyHgre5+hpktBI5396GUxyYZUvQgtUx73zlTtpfctaueXaTZkgT8a8OvS8PtjcC/ALEB38wOBa4DDiEo5Vzt7t+qf6jSbcbb+5Z7kHr2r5WSEWmxJAH/YHf/npmtBHD3fWb2RrWDgH3A/3D3R8ysD1hnZne7+1ONDFg6R9GD1DLtfYtmpU56kNqPiLRWkoD/ezM7iOAuHTN7L7Cj2kHu/iLwYvjzTjP7JUHVjwJ+lxoeDitkoPyD1LKNwnTXLtIpkgT8i4FbgSPMbA0wm6CBWmJm1g8cAzxU5rUBYABg1qzDajmtpKxce9/xWanz5pVJyYhIJ6sa8MOUzEnAAoJa/A3u/oekH2BmM4AfAJ9199dKX3f31cBqgMMPX+RJzyvNMd7ed1uFlMwV2yZ2XHYZ7NwJzwDPPAN3hD30+vrgkjordaNzlmrknCJSVpL2yL3Ap4ETCNI6D5jZVe5etb2Cme1HEOxv0JKI7Rfb3rcXBr7RH3+CnTthxozy++uVxjlFpKwkKZ3rgJ3AFeH2fwL+H3BW3EFmZgSVPL9092/EvVeaa3yd1JsrpGROL+gtM+lhqohkVZKAf6S7LyzY/omZJXnwuhT4z8B6M3s03HeJu99e6yClsqEh2Pqr8rNSl+73XHFKBgiCe3+LRicinSRJwH/EzN7r7v8OYGbvAdZWO8jdH2Si/440wfAwrLmpTOfHQ0+a2DhiTplZqSIiyQL+ccDPzCyKNIcBG8xsPeDu/q7URpdTcbNSV868ioHL+kuO0KxUEakuScA/PfVR5NjQEGx9tMo6qUXtfftbN7hSfX2VK2o66ZwiUlaSssznWzGQrIublfrsoSfB0qUTb+jU9r7dVCZ54YWwb9/k/VOnwhVXTN6fhEpIpcslucOXGtXe3jfHKZm0yjL37QuCe7n99VIJqXQ5BfwmKZ2VWv1BqmapikhrKeAnVPQgtcys1DlTtpf0ksnxXbuIdCQF/AqKZqWWPkjthYGPlmvvmwFxeeoXX6wvL37BBeBlumaYQU8P7Kjai09EmiDXAb/oQWqZWam5bO8bl6euNy/uHgT3cvvdm59rF5GychXwx2eljr4Oo3vGH6KC2vu2jdmkP7Tj+xsxdWrlf43USyWk0uUyG/ArzUpdOfMqBmbfGJRBZjElk0Rc2qZecWkbKP8alA/2kWqllWlch0iGdX3AH2/v+9sKKZlvlOslk/MHqmmUF8albepVLYWURvopjsoypct1VcCPnZXaR5mWAyIiEum4gF/0ILVSSkbtfdMTl6fetatyimXv3vTHJiIN6aiA/9Kv97Lp5l+Mby/tfa5CSqa/haPqUI1M83/lleCrnKIJByUuvDD+vI2kb8oZG0vnD4laJEhOdVTAP+qwHTx0ablKGZmk0XxyPfn2RvLi9eb3m/1cAJSLl9zqqIAvLdTsu/G0Pi/uuHrTT/VSWaZ0OQX8bhWXlqmWsujpqXynXm+p46xZ5e+ad+2qPE6IH8vMmZXPCfHpl7jr2LKlvtm9SvdIl1PA72aV0h2NpCzSSnekkZqJE3cdmt0rOdXT7gGIiEhr6A6/m9V7h1xv24G44+Ly26+8Unms9Z6zEWm1cxDpcAr43Sou911NXGfLlTGzkOtdKerTn27+ORtx4IHxzwbqoVJP6QIK+JK+POTMVeopXUABv1ullQpJI43SjhRKtetQeaXkkAJ+J4tLE6SVCkkj/ZBGCqUapVFEJlHA72RKE4hIE6ksU0QkJ1K7wzeza4Azga3ufmRanyNdIA8tCfJwjdL10kzp/BPwj8B1KX6G1KPVJYR5yKfn4Rql66UW8N192Mz60zq/NEDPBkRySQ9tO5nSBCLSRG0P+GY2AAwAHDZrVptH02GUJhCRJmp7lY67r3b3Re6+aHa5NIOIiDRF2wO+iIi0RmoB38xuBH4OLDCzLWa2Iq3PkhpFK0KVfunZgEimpVml88m0zi0N0rMBkVxSSkdEJCcU8EVEckIBX0QkJxTwRURyQgFfRCQnFPBFRHJCAV9EJCcU8EVEckIBX0QkJxTwRURyQgFfRCQnFPBFRHJCAV9EJCcU8EVEckIBX0QkJxTwRURyQgFfRCQnFPBFRHJCAV9EJCcU8EVEckIBX0QkJxTwRURyQgFfRCQnFPBFRHJCAV9EJCcU8EVEckIBX0QkJ1IN+GZ2upltMLNnzOwLaX6WiIjESy3gm9kU4ErgDGAh8EkzW5jW54mISLw07/CXAM+4+2Z33wvcBHwkxc8TEZEYaQb8PwF+U7C9JdwnIiJtMLXdAzCzAWAg3Nxl55+/oZ3jaZKDgd+1exAdSr+bePr9VKbfTXmHJ31jmgH/BeDQgu254b4i7r4aWJ3iOFrOzNa6+6J2j6MT6XcTT7+fyvS7aVyaKZ0RYJ6Zvc3M3gScDdya4ueJiEiM1O7w3X2fmf1X4MfAFOAad38yrc8TEZF4qebw3f124PY0P6NDZSpF1WT63cTT76cy/W4aZO7e7jGIiEgLqLWCiEhOKOA3mZlNMbNfmNlt7R5LpzGz58xsvZk9amZr2z2eTmJmM83s+2b2tJn90syOb/eYOoWZLQj/m4m+XjOzz7Z7XN2o7XX4GXQR8EvggHYPpEOd7O6qpZ7sW8Cd7v6JsKptersH1CncfQNwNIy3bHkBuKWtg+pSusNvIjObC3wIuLrdY5HuYWYHAsuAIQB33+vur7Z3VB1rOfArd3++3QPpRgr4zfVN4PPAWLsH0qEcuMvM1oUzrCXwNmAbcG2YDrzazN7c7kF1qLOBG9s9iG6lgN8kZnYmsNXd17V7LB3sBHc/lqCD6mfMbFm7B9QhpgLHAqvc/Rjg94DaiZcIU10fBm5u91i6lQJ+8ywFPmxmzxF0Bj3FzK5v75A6i7u/EH7fSpCDXdLeEXWMLcAWd38o3P4+wR8AKXYG8Ii7v9zugXQrBfwmcfeV7j7X3fsJ/tl5n7v/ZZuH1THM7M1m1hf9DJwGPNHeUXUGd38J+I2ZLQh3LQeeauOQOtUnUTqnIarSkVY5BLjFzCD47+6f3f3O9g6po1wI3BCmLTYD57Z5PB0lvEk4FTi/3WPpZpppKyKSE0rpiIjkhAK+iEhOKOCLiOSEAr6ISE4o4IuI5IQCvmSSmb2/XMfSSvub8HkfNbOFBdv3m1ns+qvhWHaYWcOLBJnZ/mEnyb1mdnCj55NsUsAXaY6PAgurvmuyB9z9g41+uLu/7u5HA79t9FySXQr40hbhzNt/M7PHzOwJM/uLcP9xZvbTsMHaj83sLeH++83sW+Fd7BNmtiTcv8TMfh42HftZwWzVpGO4xsweDo//SLj/HDP7oZndaWabzOxrBcesMLON4TH/18z+0czeR9Dj5evh+I4I335W+L6NZnZiwjH9z3DNgMfM7CsF1/4PZrY27JW/OBzfJjP7ctLrFdFMW2mX04HfuvuHIGgRbGb7AVcAH3H3beEfgb8HzguPme7uR4dN164BjgSeBk50931m9gHgMuDjCcdwKUELjPPMbCbwsJndE752NHAMsAfYYGZXAG8A/4ugz81O4D7gMXf/mZndCtzm7t8PrwdgqrsvMbMPAl8EPhA3GDM7A/gI8B53321mswrjOsnFAAACIUlEQVRe3uvui8zsIuBfgeOAV4Bfmdk/uPv2hNcsOaaAL+2yHrjczL5KECgfMLMjCYL43WHAnAK8WHDMjQDuPmxmB4RBug/4rpnNI2i/vF8NYziNoOHd58LtXuCw8Od73X0HgJk9BRwOHAz81N1fCfffDMyPOf8Pw+/rgP4E4/kAcK277waIPid0a/h9PfCku78YjmEzcCiggC9VKeBLW7j7RjM7Fvgg8GUzu5egg+aT7l5peb/SPiAO/B3wE3f/mJn1A/fXMAwDPh6uqDSx0+w9BHf2kTeo7/+V6Bz1Hl/uXGMUj22sCeeWnFAOX9rCzN4K7Hb364GvE6RJNgCzo/VczWw/M3tnwWFRnv8EYEd4B34gwZJ3AOfUOIwfAxda+M8JMzumyvtHgJPM7I/MbCrFqaOdBP/aaMTdwLlmNj0cz6wq7xepiQK+tMtRBDnzRwny2192973AJ4CvmtljwKPA+wqOGTWzXwBXASvCfV8DBsP9td7p/h1BCuhxM3sy3K4o7Od/GfAwsAZ4DtgRvnwT8Nfhw98jyp8hXtg99FZgbfh7+VyVQ0Rqom6Z0hXM7H7gc+6+ts3jmOHuu8I7/FuAa9y9rgW1zez9BNd0ZhPH9xywSAvFSzm6wxepzd+Ed99PAM8CP2rgXHuBI5s58YrgXyxaU1nK0h2+iEhO6A5fRCQnFPBFRHJCAV9EJCcU8EVEckIBX0QkJxTwRURy4v8DTmny/Tu5SnUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f515084db10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_decision_region(X, y, classifier=ppn)\n",
    "plt.xlabel('sepal length [cm]')\n",
    "plt.ylabel('petal length [cm]')\n",
    "plt.legend(loc='upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "虽然对于Iris数据集，感知机算法表现的很完美，但是\"收敛\"一直是感知机算法中的一大问题。Frank Rosenblatt从数学上证明了只要两个类别能够被一个线性超平面分开，则感知机算法一定能够收敛。然而，如果数据并非线性可分，感知计算法则会一直运行下去，除非我们人为设置最大迭代次数n\\_iter。 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 自适应线性神经元及收敛问题\n",
    "\n",
    "\n",
    "本节我们学习另一种单层神经网络：自适应线性神经元(ADAptive LInear NEuron, 简称Adaline)。在Frank Rosenblatt提出感知机算法不久，Bernard Widrow和他的博士生Tedd Hoff提出了Adaline算法作为感知机的改进算法(B.Widrow et al. Adaptive \"Adaline\" neuron using chemical \"memistors\".)\n",
    "\n",
    "\n",
    "相对于感知机，Adaline算法有趣的多，因为在学习Adaline的过程中涉及到机器学习中一个重要的概念：最小化损失函数。学习Adaline为以后学习更复杂高端的算法(比如逻辑斯蒂回归、SVM等)起到抛砖引玉的作用。\n",
    "\n",
    "Adaline和感知机的一个重要区别是Adaline算法中权重参数更新按照线性函数而不是单位阶跃函数。当然，Adaline中激活函数也简单的很，$$\\phi(z)=\\phi(w^{T}x)=w^{T}x$$\n",
    "\n",
    "虽然Adaline中参数更新不是使用阶跃函数，但是在对测试集样本输出预测类别时还是使用阶跃函数，毕竟要输出离散值-1,1。\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/11/575bcd008be3f.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### 使用梯度下降算法更新参数\n",
    "\n",
    "有监督的机器学习(supervised machine learning)算法中，一个重要的概念就是定义目标函数(objective function)，而目标函数就是机器学习算法在学习过程中要优化的目标，目标函数也常被称为损失函数(cost function/loss function)。训练过程就是找到能使目标函数值最小的参数权重值。\n",
    "\n",
    "对于Adaline算法，我们定义损失函数为样本真实值和预测值之间的误差平方和(Sum of Squared Erros, SSE):\n",
    "\n",
    "$$\\mathrm{J}(\\mathrm{w}) = \\frac{1}{2}\\sum_{i}(y^{(i)}-\\phi(z^{(i)}))^{2}$$\n",
    "\n",
    "\n",
    "上式中的系数$\\frac{1}{2}$完全是为了求导数方便而添加的，没有特殊的物理含义。相对于感知机中的单位阶跃函数，使用线性函数作为激活函数的一大优点是Adaline的损失函数是可导的。另一个很好的特性是Adaline的损失函数是凸函数，这样我们就可以使用传说中鼎鼎大名的的优化算法：梯度下降(gradient descent)来找到使损失函数取值最小的权重参数。\n",
    "\n",
    "\n",
    "\n",
    "如下图所示，我们可以把梯度下降算法看做\"下山过程\"，直到遇到局部最小点或者全局最小点才会停止计算。在每一次迭代过程中，我们沿着梯度下降方向迈出一步，而步伐的大小由学习率和梯度大小共同决定。\n",
    "\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/11/575cc3c888991.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "使用梯度下降算法，实质就是运用损失函数的梯度来对参数进行更新：\n",
    "\n",
    "$$\\mathrm{w} := \\mathrm{w} + \\Delta \\mathrm{w}$$\n",
    "\n",
    "此时，$\\Delta w$的值由负梯度乘以学习率$\\eta$确定：\n",
    "\n",
    "$$\\Delta w = -\\eta \\Delta J(w)$$\n",
    "\n",
    "而要计算出损失函数的梯度，我们需要计算损失函数对每一个权重参数的偏导数$w_{j}$:\n",
    "\n",
    "\n",
    "\n",
    "$$\\frac{\\partial J}{\\partial w_{j}}=-\\sum_{i}(y^{(i)}-\\phi(z^{(i)}))x_{j}^{(i)}$$\n",
    "\n",
    "因此，$\\Delta w_{j} = -\\eta \\frac{\\partial J}{\\partial w_{j}}=\\eta \\sum_{i}(y^{(i)}-\\phi(z^{(i)}))x_{j}^{(i)} $.\n",
    "\n",
    "注意所有权重参数还是同时更新的，所以Adaline算法的学习规则可以简写: $\\bf{w}:=w+\\Delta w$.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "虽然简写以后的学习规则和感知机一样，但不要忘了$\\phi(z)$的不同。此外，还有一点很大的不同是在计算$\\Delta w$的过程中：Adaline需要用到所有训练集样本才能一次性更新所有的w，而感知机则是每次用一个训练样本更新所有权重参数。所以梯度下降法常被称为批量梯度下降 (**batch** gradient descent)。\n",
    "\n",
    "\n",
    "**福利：**\n",
    "详细的损失函数对权重的偏导数计算过程为：\n",
    "\n",
    "$$\\frac{\\partial J}{\\partial w_{j}}=\\frac{\\partial \\frac{1}{2}\\sum_{i}(y^{(i)}-\\phi (z^{(i)}))^{2}}{\\partial w_{j}}$$\n",
    "\n",
    "$$=\\frac{1}{2}\\cdot \\frac{\\partial \\sum_{i}(y^{(i)}-\\phi (z^{(i)}))^{2}}{\\partial w_{j}}$$\n",
    "\n",
    "$$=\\frac{1}{2}\\sum_{i}2\\cdot (y^{(i)}-\\phi(z^{(i)}))\\frac{\\partial (y^{(i)}-\\phi(z^{(i)}))}{\\partial w_{j}}$$\n",
    "\n",
    "\n",
    "$$=\\sum_{i}(y^{(i)-\\phi(z^{(i)})})\\frac{\\partial (y^{(i)}-\\sum_{i}(w_{j}^{(i)}x_{j}^{(i)}))}{\\partial w_{j}}$$\n",
    "\n",
    "$$=\\sum_{i}(y^{(i)}-\\phi(z^{(i)}))(-x_{j}^{(i)})$$\n",
    "\n",
    "$$=-\\sum_{i}(y^{(i)}-\\phi(z^{(i)}))x_{j}^{(i)}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Python实现自适应线性神经元\n",
    "\n",
    "既然感知机和Adaline的学习规则非常相似，所以在实现Adaline的时候我们不需要完全重写，而是在感知机代码基础上进行修改得到Adaline，具体地，我们需要修改fit方法，实现梯度下降算法:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "class AdalineGD(object):\n",
    "    \"\"\"ADAptive LInear NEuron classifier.\n",
    "\n",
    "    Parameters\n",
    "    ----------------\n",
    "    eta: float\n",
    "    Learning rate (between 0.0 and 1.0)\n",
    "    n_iter: int\n",
    "    Passes over the training dataset.\n",
    "\n",
    "    Attributes:\n",
    "    ------------------\n",
    "    w_: 1d-array\n",
    "    Weights after fitting.\n",
    "    errors_: int\n",
    "    Number of misclassification in every epoch.\n",
    "\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self, eta=0.01, n_iter=50):\n",
    "        self.eta = eta\n",
    "        self.n_iter = n_iter\n",
    "\n",
    "    def fit(self, X, y):\n",
    "        \"\"\"Fit training data.\n",
    "\n",
    "        Parameters\n",
    "        ---------------\n",
    "        X: {array-like}, shape=[n_samples, n_features]\n",
    "        Training vectors,\n",
    "        y: array-like, shape=[n_samples]\n",
    "        Target values.\n",
    "\n",
    "        Returns\n",
    "        -----------\n",
    "        self: object\n",
    "        \"\"\"\n",
    "        self.w_ = np.zeros(1 + X.shape[1])\n",
    "        self.cost_ = []\n",
    "\n",
    "        for i in range(self.n_iter):\n",
    "            output = self.net_input(X)\n",
    "            errors = (y - output)\n",
    "            self.w_[1:] += self.eta * X.T.dot(errors)\n",
    "            self.w_[0] += self.eta * errors.sum()\n",
    "            cost = (errors ** 2).sum() / 2.0\n",
    "            self.cost_.append(cost)\n",
    "        return self\n",
    "\n",
    "    def net_input(self, X):\n",
    "        \"\"\" Calculate net input\"\"\"\n",
    "        return np.dot(X, self.w_[1:]) + self.w_[0]\n",
    "\n",
    "    def activation(self, X):\n",
    "        \"\"\" Compute linear activation\"\"\"\n",
    "        return self.net_input(X)\n",
    "\n",
    "    def predict(self, X):\n",
    "        \"\"\" Return class label after unit step\"\"\"\n",
    "        return np.where(self.activation(X) >= 0.0, 1, -1)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "不像感知机那样每次用一个训练样本来更新权重参数，Adaline基于整个训练集的梯度来更新权重。\n",
    "\n",
    "**注意，X.T.dot\\(errors\\)是一个矩阵和向量的乘法，可见numpy做矩阵计算的方便性。**\n",
    "![](https://ooo.0o0.ooo/2016/06/15/576205a8994a9.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在将Adaline应用到实际问题中时，通常需要先确定一个好的学习率$\\eta$这样才能保证算法真正收敛。我们来做一个试验，设置两个不同的$\\eta$值：$\\eta = 0.01, \\eta=0.0001$。然后将每一轮的损失函数值画出来，窥探Adaline是如何学习的.\n",
    "\n",
    "\\(学习率$\\eta$,迭代轮数n\\_iter也被称为超参数\\(hyperparameters\\),超参数对于模型至关重要，在第四章我们将学习一些技巧，能够自动找到能使模型达到最好效果的超参数。\\)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfIAAAEWCAYAAACdRBVEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzs3Xd4VHXWwPHvSQ81IDWhChoEEQKhqa9iBXVVxLIiKkVFdN3V1cWV3bWt66IbdVfXipSg2FAR64IVu0AgQGgRpIdeAiEkkHLeP+YGB0iZlMmdmZzP88zDzG1zhuTkzL33V0RVMcYYY0xwCnM7AGOMMcZUnRVyY4wxJohZITfGGGOCmBVyY4wxJohZITfGGGOCmBVyY4wxJohZIS+HiIwUke983PYhEZnuPG8nIgdEJNy/EQYuEfmLiExyOw5Td1i+Vp3la3Crk4VcROaKyF4RifbH8VV1o6o2UNUifxy/PCKSKiL/qO33PZaq/lNVb3Y7Djj6j3Y1jvFHEdkmIvtFZEp5vzsicp6IrBKRgyLylYi091p3jYj84KybW52Y6grLV/+zfC0zX6Od/fc7x7u7EvvWWq7XuUIuIh2A/wMUuMzVYIKUiES4HUOJ2ohFRAYB9wHnAe2BE4GHy9i2GTATuB9oCqQBb3ltsgf4D/CYH0MOGZav1Wf5Wq18fQg4yTnOOcC9IjLYx31rLdfrXCEHbgR+AlKBEd4rROQEEfnA+fY1H+h0zPqnRWSTs36hiPxfaW8gIh1EREt+aZ0zikdE5HsRyRGRT51fgpLt+zvf3LJFZImIDKzZj3zkfbqIyGciskdEMkXkGq91l4hIuvPZNonIQ6V8nptEZCPwpdeyESKyUUR2ichfvfbxvnRZ0baxIjLNOetaKSL3isjmcj6HisjvRGQ1sNpZVurPxkm6vwC/Fc/l0yXO8sYiMllEtopIloj8Q8q+tDoCmKyqy1V1L/AIMLKMbYcCy1X1bVXNx/OHoIeIdAFQ1c9VdQawpazPZ45i+Wr56lq+Osd6RFX3qupK4GWvYwVMrtfVQv6a8xgkIi291j0H5AOtgdHOw9sCoCeeb1+vA2+LSIyP73sdMApoAUQBfwIQkQTgY+AfznH/BLwrIs0r/cnKISL1gc+cuFsA1wLPi0hXZ5NcPP83ccAlwG0iMuSYw5wNnAIM8lp2JpCI59vvAyJySjlhlLXtg0AHPN+cLwCu9+EjDQH6ASXxl/qzUdXZwD+Bt5zLpz2c7VOBQqAzkARcCJR1abEbsMTr9RKgpYicUNG2qpoL/OIsN5Vn+Wr5Ci7kq4g0wfO7deyxulW0bxlx+U2dKuQiciaeSyQzVHUhnv/065x14cCVwAOqmquqy4Bp3vur6nRV3a2qhar6JBCN5xfdF1NV9WdVzQNm4PklBk8SfKKqn6hqsap+hucSzcXV+7TH+Q2wXlWnOvGnA+8CVwOo6lxVzXBiWAq8gecPgbeHnP+bPK9lD6tqnqouwfNL3YOylbXtNcA/nW+9m4FnfPg8E1R1T0kslfnZOMXgYuAu5/PsAP6N549laRoA+7xelzxv6MO2JduXtq0ph+Wr5Su4mq8Njtnfe11F+9aqOlXI8Vwm+VRVdzmvX+fXy3XNgQhgk9f2G7x3FpE/OZeS9olINtAYaIZvtnk9P8ivvyTtgaudy3TZznHPxPNN8CgiMty53HRARP7n4/uWaA/0O+Z9hgOtnGP3E09jjZ0isg8YW8pn28TxyvpcpSlr2/hjjl3a+xzrqG0q+bNpD0QCW73+L17Cc+ZTmgNAI6/XJc9zfNi2ZPvStjXls3y1fAX38vXAMft7r6to31oVMI0g/E1EYvF8kwwXkZJf0GggTkR6AMvwXLppC6xy1rfz2v//gHvxXGZarqrFIrIXkGqGtgl4VVVvqWhDVS25xFjV9/laVS8oY/3rwLPARaqaLyL/4fjE8tdUeVuBNsAK53VbH/Y5EosPP5tj494EHAKaqWqhD++1HM/ZyAzndQ9gu6ruLmPbI/dynUuknZzlxkeWr5avXlzJV1XdKyJbnf0/8zrW8or29SHGGlWXzsiHAEV47tH0dB6nAN8CNzpdT2YCD4lIPedelHfjmoZ4/nDsBCJE5AGO/zZWFdOBS0VkkIiEi0iMiAwUkTbVOGbJcUoeUcBHwMkicoOIRDqPPl73vRoCe5w/Cn1xLmHWkhnAeBFp4tyDvKOS+1f0s9kOdBCRMABV3Qp8CjwpIo1EJExEOonIsZcmS7wC3CQiXUUkDvgbnnt2pXkPOFVErnTuxz4ALFXVVeC5JOwsjwDCnJ9PZCU/b11g+Wr56nq+Osf6m/NZuwC3eB0rYHK9LhXyEXjue21U1W0lDzzfaoeLp8XqHXguH23D88Oa6rX/HGA28DOeS3j5+HZJqVyqugm4HE9LzZ3OMcdRvZ/NfUCe1+NLVc3B00DkWjytKLcBj+M5ywG4Hfi7iOTg+YWccexB/ejvwGZgHfA58A6eb+C+quhn87bz724RWeQ8vxFPI6YVwF7nPY+7PArgNMD5F/AVsNF5jwdL1ovIchEZ7my7E8+920ed4/bj6Ht5N+D5mbyAp1tVHp6WsOZolq+Wr4GQrw/iaZuxAfgaSHGOH1C5Lqr+uvpiTNWIyG3Atapa1jduY0yAsHx1X106IzcBSkRai8gZziWzROAePJetjDEBxvI18NSZxm4moEXhaYXaEcgG3gSedzUiY0xZLF8DjF1aN8YYY4KY3y6tOy305otnCMPlIvKws7yjiMwTkTUi8pbTQtMYY4wxVeC3M3IREaC+qh5wmtx/B9wJ3A3MVNU3ReRFYImqvlDesZo1a6YdOnTwS5zGhJKFCxfuUtUaHS60JlkuG+ObyuSy3+6Rq+cbQsnIOJHOQ4Fz+bXP4zQ8A82XW8g7dOhAWlqafwI1JoSIyIaKt3KP5bIxvqlMLvu11brTIX4xsAPPyDi/ANleo/NsBhLK2HeMiKSJSNrOnTv9GaYxxhgTtPxayFW1SFV74hnOry/QpYJdvPedqKrJqprcvHnAXik0xhhjXFUr/chVNRvPKDsD8IyVXHJJvw2QVRsxGGOMMaHIn63Wmzvj3JZMgHABsBJPQb/K2WwE8L6/YjDGlE9EpojIDhFZVsb6cSKy2HksE5EiEWnqrBssIplOD5T7ajdyY0wJf56Rtwa+EpGleCaR/0xVPwL+DNwtImuAE4DJfozBGFO+VGBwWStVNUVVezq3yMbjmZFrj3jmA38OuAjPxCbDnIlLjDG1zJ+t1pcCSaUsX4vnfrkxxgez0rNImZPJluw84uNiGTcokSFJpbYRrTRV/UZEOvi4+TDgDed5X2CNk8+IyJt4JhNZUca+PvHnZzUmVNlY68YEsFnpWYyfmUFWdh4KZGXnMX5mBrPSa7dpiYjUw3Pm/q6zKIGjZ6yqdg+UQPmsxgQbK+TGBLCUOZnkFRQdtSyvoIiUOZm1HcqlwPequqeyO/raAyWAPqsxQcUKuTEBbEt2XqWW+9G1/HpZHTy9Tdp6va52D5QA+qzGBBUr5MYEsIYxpTdjiY+LrbUYRKQxcDZH9zBZAJzkzJ0QhafQf1Cd9ynrM9XmZzUmGFkhNyZAvbtwM/vzCwkXOWp5bGQ44wYl1sh7iMgbwI9AoohsFpGbRGSsiIz12uwK4FNVzS1Z4IzOeAcwB0+30hmqurw6sYwblEhsZPhRy2Ijw2rssxoTqmw+cmMC0Lerd/Lnd5dyeqcTGJqUwL8/X+2vVuvDfNgmFU83tWOXfwJ8UiOBwJHPlDInkyzncvplPeOt1boxFbBCbkyAWbFlP7dNX0TnFg148YbeNIqJ5KrkthXvGAKGJCUwJCkBVeXKF37gi5U7yckvoGFMpNuhGROw7NK6MQEkKzuPUanzaRAdwdRRfWhURwuYiPDgpd3YdeAQz331i9vhGBPQrJAbEyD2HSxg5JT5HDxUROroPrRuXLcbefVoG8fQXglM+W4dG3cfdDscYwKWFXJjAsChwiLGvJrG+t25vHRjb7q0auR2SAHh3kFdCA8TJvxvpduhGBOwrJAb47LiYuWeGUuYt24PT1zdg9M7NXM7pIDRqnEMtw3sxP+WbeOntbvdDseYgGSF3BiXPTZ7FR8t3cp9F3Xh8p7WQvtYt/zficQ3juGRj1ZQVKxuh2NMwLFCboyLpn6/jonfrOXGAe259awT3Q4nIMVGhfPni7qwfMt+3l242e1wjAk4VsiNccnsZVv5+0cruKBrSx68tBtyzMAv5leX9YinV7s4/jUnk5z8ArfDMSagWCE3xgVp6/dw55uL6dk2jmeuTSI8zIp4eUSEB5zuaM/Pte5oxnizQm5MLftl5wFufiWN+LhYJo/oQ2xUeMU7GXq2jWNoUgKTv13Hpj3WHc2YElbIjalFO3LyGTFlPuEipI7qQ9P6UW6HFFTGDU607mjGHMMKuTG1JPdQITelprH7wGGmjOxD+xPqux1S0GndOJaxZ3fik4xtzLPuaMYAVsiNqRWFRcX87vVFLN+yj2evS6JH2zi3QwpaY87ydEf7u3VHMwawQm6M36kqf5u1jLmZO3lkyKmcd0pLt0MKakd1R1tk3dGMsUJujJ/998s1vLlgE3ec05nh/dq7HU5IuKxHPEnt4kiZk8mBQ4Vuh2OMq6yQG+NHb6dt4qnPfmZorwTuufBkt8MJGSLCA7/pys6cQzz/1Rq3wzHGVTYfuTE1aFZ6FilzMtmSnUfT+lHsyT3M/53UjMeGnmYDvtSwpHZNuCIpgUnfrWNY33a0bVrP7ZCMcYWdkRtTQ2alZzF+ZgZZ2XkosDv3MACXdG9FVISlmj/cOziRMIHH/rfK7VCMcY39dTGmhqTMySSvoOioZQr890sbicxfSrqjfZyxlfnr9rgdjjGusEJuTA3Zkp1XqeWmZtx6VidaN47h7x8tp9i6o5k6yAq5MTWkdeOYUpfHx8XWciR1S2xUOPdd1IVlWdYdzdRNfivkItJWRL4SkRUislxE7nSWPyQiWSKy2Hlc7K8YjKktxcVKi4bRxy2PjQxn3KBEFyKqW0q6o/3LuqOZOsifZ+SFwD2q2hXoD/xORLo66/6tqj2dxyd+jMGYWvHPT1ayePM+Lu8RT0JcLAIkxMUyYWh3hiQluB1eyBMR7ne6o70w17qjmbrFb93PVHUrsNV5niMiKwH7i2ZCzuTv1jHpu3WMPL0DD17a1bqZuaRXuyYM6RnPy9+u49o+1h3N1B21co9cRDoAScA8Z9EdIrJURKaISJPaiMEYf/gkYyv/+HgFg7u14v7fWBF3272Du3i6o8227mim7vB7IReRBsC7wF2quh94AegE9MRzxv5kGfuNEZE0EUnbuXOnv8M0ptLmr9vDXW8tple7Jvzn2p6Eh1kRd1t8nNMdbelWFqy37mimbvBrIReRSDxF/DVVnQmgqttVtUhVi4GXgb6l7auqE1U1WVWTmzdv7s8wjam0NTtyuOWVNNrExTLpxmRiIsPdDsk4jnRH+3CFdUczdYI/W60LMBlYqapPeS1v7bXZFcAyf8VgjD/s2J/PiCkLiAwXpo3uS5P6UW6HZLzERoXz58FdyMjaZ93RTJ3gzzPyM4AbgHOP6Wr2LxHJEJGlwDnAH/0YgzE16sChQkalLmDvwcNMGdnHGlQFqMt6xNOzrac7Wq51RzMhzp+t1r8DSrtpaN3NTFAqKCrm9tcWsWpbDpNGJHNamzi3QzJlCAsTHri0K0Of/4H+//yCA4cKiY+LZdygROsOaEKOzX5mjA9Ulb/MzOCbn3fy+JXdOSexhdshmQps3H2QcBFynDPyrOw8xs/MALBibkKKDdFqjA/+8/lq3l64mTvPO4nf9mnndjjGBylzMinSoxu75RUUkTIn06WIjPEPK+TGVOCtBRt5+ovVXN27DXedf5Lb4Rgf2SQ2pq6wQm5MOb7K3MFf3lvGWSc3559Du4fcgC/OoEw7RKTM3iMiMtBprLpcRL72Wv5HZ9kyEXlDREqfNcYlZU1WY5PYmFBjhdyYMmRs3sfvXltEl1YNeX54LyLDQzJdUoHBZa0UkTjgeeAyVe0GXO0sTwD+ACSr6qlAOHCt36OthHGDEoktpX//6DM71H4wxvhRSP5lMqa6Nu05yKjUBTSpF8XUkX1oEB2a7UJV9RugvCHQrgNmqupGZ/sdXusigFgRiQDqAVv8FmgVDElKYMLQ7kcmsWnRMJroCOGdhVkcPGxd0kzoCM2/TsZUw97cw4yYOp+ComLeHNOPFo0C6opxbTsZiBSRuUBD4GlVfUVVs0TkCWAjkAd8qqqflnYAERkDjAFo1652GwoOSUo4qoX63MwdjE5dwLi3l/LsdUkhd6vE1E12Rm6Ml/yCIm5+JY3Ne/OYNCKZzi0auh2S2yKA3sAlwCDgfhE52Zns6HKgIxAP1BeR60s7QCANtzwwsQV/HtyFjzO28vzcX1yNxZiaUqkzchGpD+SrapGf4jHGNUXFyl1vLmbRxr08d10v+nRo6nZIVVLDeboZ2K2quUCuiHwD9HDWrVPVnc57zgROB6bXwHv61ZizTmTF1v088WkmiS0bcn7Xlm6HZEy1lHtGLiJhInKdiHwsIjuAVcBWEVkhIiki0rl2wjTGv1SVRz5awezl2/jbJV25uHvrincKEMXFxbz++utccskl4CmyNZmn7wNnikiEiNQD+gEr8VxS7y8i9Zx5Fc5zlgc8EeHxK0+jW3wj7nprMWt25LgdkjHVUtGl9a/wTDk6Hmilqm1VtQVwJvAT8HhZl9OMCQaz0rM447Ev6Tj+E1J/WM/ZJzfjpjM7uh1WpZxzzjn88ssvTJgwAWBJZfJURN4AfgQSRWSziNwkImNFZCyAqq4EZgNLgfnAJFVdpqrzgHeARUAGnr8lE/37SWtOTGQ4E29IJiYyjFteWci+vAK3QzKmykS17Gn+RCRSVcv9Dfdlm+pKTk7WtLQ0f76FqYNmpWcxfmYGeQW/XoGOjQxjwtDTgmoIz4KCAiIjIwEQkYWqmnzsNrWRp74ItFxesH4P1738E6d3asaUkX1sTnkTMMrK5dKUe0auqgUiEi4iq8rbprIBGhMIUuZkHlXEAfIKioNuCM/IyEiKioro0qVLmdtYnpauT4emPHzZqXz9807+NafMP3PGBLQKW607DWYyRcQGmDYhJZSG8AwPDycxMRHAJkevpOv6teP6/u146eu1vL84y+1wjKk0X1utNwGWi8h8ILdkoape5peojPGz7fvzCRMoKuXOUrAO4bl3716AbiLyBZanlfLAb7rx8/YD3PvOUk5s1oDubRq7HZIxPvO1kN/v1yiMqUU5+QWMnLqA8DAhQoRDhcVH1sVGhjNuUKKL0VXdI488wsCBA1cDf3c7lmATFRHG88N7cfmz3zPm1TQ+uONMmjeMdjssY3zi04Awqvo1ni4tDZ3HSmeZMUHlcGExt01fxOrtObw8og+PX3nakSE8E+JimTC0e1A1dPN29tlnA+RjeVolzRpE89INvdl78DC3TV/IYa8veMYEMp/OyEXkGiAFmAsI8F8RGaeq7/gxNmNqlKpy38ylfLdmFylXncbZJ3tGGQvWwn2sGTNmAJyCZ2ITy9MqODWhMSlX9eD3b6Tz4AfLmTC0u9shGVMhXy+t/xXoUzJhgog0Bz7H04/UmKDw1Gc/M3NRFndfcDJXJ7d1O5wa9+ijj4LnLHwEWJ5W1aU94lmxdT8vzP2FbvGNuL5/e7dDMqZcvo61HnbMrEe7K7GvMa57fd5G/vvlGq7t05bfnxuaAxIWFxcDeE/rZXlaRX+6MJFzEpvz0AfLmbd2t9vhGFMuX5N8tojMEZGRIjIS+Bj4xH9hGVNzvli5nb/NyuCcxOb8Y8ipITvj1eDBgwFOsjytvvAw4elhSbQ7oR63v7aIzXsPuh2SMWXytbHbOOAl4DTnMVFV/+zPwIypCUs2ZXPH6+l0i2/Ms9f1IiI8dE9QU1JSAHZieVojGsVE8vKNyRwuLGbMKwvJO2xzRZnAVOE9chEJBz5X1XOAmf4PyZiasWF3LqNTF9CsYRRTRvahfnSlJvsLKkVFRZx//vkA2ap6t9vxhIpOzRvwzLAkRk9bwLh3lvDfYTaHuQk8vo7sViwiNkKCCRp7cg8zcuoCilRJHdU35PsEh4eHExYWBhDudiyh5pwuLRg3KJGPlm7lha9tDnMTeHw9RTkAZIjIZxw9YtQf/BKVMdWQd7iIm6YtYEt2Hq/f0o9OzRu4HVKtaNCgAUBXEZmM5WmNuu3sTqzYsp+UOZlk5x7m44xtbMnOIz4ulnGDEkOmC6MJTr4W8pnYZXUTBIqKlTvfTGfxpmxeGN6L3u2buh1SrRk6dCgffPDBFuAbt2MJNSJCylU9SN+4l4nfrjuyPCs7j/EzM4DQGY/ABB9f75FfqKrDayEeY6pMVXn4w+V8umI7D13alcGntnY7pFpTVFTEp59+CrBbVae5HU8oio0Kp7D4+MH58wqKSJmTaYXcuMbXe+TtRcRmVTIBbeI3a3nlxw2MOetERp7R0e1walV4eDgbNmwAz4huxk927D9U6vJgnDHPhA5fL62vBb4XkQ84+t7bU2XtICJtgVeAloDi6QrztIg0Bd4COgDrgWtUdW+VojfG8f7iLCb8bxW/Oa019w0ue17uUHbiiSfy/fffdxGR+/ExT03lxMfFklVK0Q7WGfNMaPC1U+0vwEfO9g29HuUpBO5R1a5Af+B3ItIVuA/4QlVPAr5wXhtTZT/8sos/vb2Efh2b8uQ1PQgLq5snpZ06dQLIpnJ5aiph3KBEYiOP7hgQJvDH809yKSJjQFRLmZDZlx1FIlS1sOItj2z/PvCs8xioqltFpDUwV1XLnTcyOTlZ09LSqhSnCW2rtu3n6hd+pFXjGN4ZezqN60W6HZKrRGShqiZ7va5UnvpbKOTyrPQsUuZksiU7j8axkWTnFTCoW0ueC/EBh0ztOjaXy1Pub52IfOf1/NVjVs+vREAdgCRgHtBSVbc6q7bhufRe2j5jRCRNRNJ27tzp61uZOmTrvjxGTV1AbFQ4qaP71tkifuaZZ3q/PLZxgM95anwzJCmB7+87l3WPXcLiBy/kgd90Zc7y7dw9YwlFpTSGM8bfKrpHXt/r+anHrPPp+qWINADeBe5S1f3eoyKpqopIqb/5qjoRmAieb/G+vJepO/bnFzBq6gJy8gt569b+JNThe5S5ubneL4/9j6ib9xlq0egzO5JfWMS/ZmcSHRHG41eeVmdv7xh3VFTItYznpb0+johE4inir6lqST/07SLS2uvS+o6yj2DMr7wvaUZFhHG4sJhXbupLt/i6PejgMUOGVjpPTfXdPrAz+QXFPPPFamKjwnn4sm42lKupNRUV8jgRuQLPJfg4ERnqLBeg3L+e4vktnoxnfmTvVrMfACOAx5x/369K4KZumZWexfiZGeQVeCauOFRYTGS4sPvAYZcjc192djbvvfdeyTSmEZXJU1Nz/nj+SeQXFDHxm7XERIYz/qIuVsxNraiokH8NXOb1/FKvdRWNHnUGcAOeoV0XO8v+gqeAzxCRm4ANwDWVitjUSSlzMo8U8RIFRWoDcQBnn302H3zwQcnLHCqXp6aGiAjjL+pC3mFPMY+NDOePF5zsdlimDii3kKvqqKoeWFW/o+z7c+dV9bimbiprwA0biAOmTp165Hlqaur66uStqR4R4eHLunGosIinv1hNTGQ4tw3s5HZYJsRVuq+EiHzkj0CMKU+T+qUPLGgDcZTO8tQ9YWHChKGncVmPeB6fvYqp36+reCdjqqEqEzTX7euYptalb9zL/rzDiID3sAexkeGMG1TuEAR1meWpi8LDhCev6cGhwiIe/nAFMZHhDOvbzu2wTIiqyugF6TUehTFlWL8rl5umpREfV4+/X96NhLhYBEiIi2XC0O51/v54OSxPXRYZHsYzw5IYmNicv7yXwXvpm90OyYSoSp+Rq+pofwRizLF2HzjEiKnzUVWmje5Lx2b1uaF/B7fDCgqWp4EhOiKcF6/vzejUBdwzYwnREeFc3L3uzMpnake5hVxEMiinH6qqnlbjERkD5B0uYvS0NLbty+eNMf3p2Kx+xTvVUd27d/fu5tRVRJZ6r7c8dVdMZDgv35jMiCnz+cMb6URHhHHeKaUOaGlMlVR0af03eLqyzHYew53HJ87DmBpXWFTM799YRMbmbP47LIle7Zq4HVJA++ijj/jwww8ZPHgwwH4sTwNO/egIpozqQ9f4Rtw2fRHfrrZhp03NKbeQq+oGVd0AXKCq96pqhvO4D7iwdkI0dYmq8tCHy/l85Q4euqwbF3Zr5XZIAa99+/a0b9+ezz77DGCz5WlgahQTySuj+3Ji8/rc8koa89budjskEyJ8bewmInKG14vTK7GvMT574etfmP7TRsae3YkbB3RwO5yg4sxkeOQehOVp4ImrF8X0m/uREBfL6NQFpG/c63ZIJgT4muQ3Ac+LyHoRWQ88D1hjGlOj3kvfzL9mZ3J5z3jutW5llTZ58mSA9pXJUxGZIiI7RGRZOdsMFJHFIrJcRL72Wh4nIu+IyCoRWSkiA2roo4S0Zg2ief2W/jRrGM2IKfNZlrXP7ZBMkKvUfOQi0hhAVWv1Ny8U5jA25ft+zS5GTp1PcvumpI7uQ3REuNshBSURWYgzcqIveSoiZwEHgFdU9dgZDhGROOAHYLCqbhSRFqq6w1k3DfhWVSeJSBRQT1Wzy3s/y+Vfbd57kGte/JF9eQXUj45gZ84h4uNiGTco0bpVmpqbj9zrgC1FZDLwpqruE5GuzljpxlTbyq37GfvqQk5s1oAXb+htRbyKtm/fDtCeSuSpqn4D7Clnk+uAmaq60dm+pIg3Bs7CMzESqnq4oiJujtamST1GndGR3MNF7Mg5hAJZ2XmMn5nBrPQst8MzQcTXS+upwBwg3nn9M3CXPwIydcuW7DxGTV1A/egIpo7qQ+PYSLdDClojR44ET6v1mszTk4EmIjJXRBaKyI3O8o7ATmCqiKSLyCQRKbWPoIiMEZE0EUnbudNaa3tL/WH9ccvyCopImZNZ+8GYoOVrIW+mqjOAYgBVLQSKyt/FmPLtyytg5NT55B4qJHV0Hxs3vZp27doFsJeazdP0274NAAAgAElEQVQIoDdwCTAIuF9ETnaW9wJeUNUkIBe4r7QDqOpEVU1W1eTmzZtXM5zQYpMBmZrgayHPFZETcAaHEZH+gLXQMFV2qLCIW19NY92uXF66oTddWjVyO6SgV79+fYBwajZPNwNzVDVXVXfhmRa1h7N8s6rOc7Z7B09hN5VQ1pfX5g2jazkSE8x8LeR3Ax8AnUTke+AV4Pd+i8qEtOJiZdzbS/lp7R5SrurB6Z2buR1SSHjqqacATqJm8/R94EwRiRCRekA/YKWqbgM2iUhJ94LzgBXVfK86Z9ygRGIjj28TkpNfwNLN1uTA+KbCsdZFJAyIAc4GEvHMMZ6pqgV+js2EqH/NyeSDJVu4d7C1zq0pxcXF5OfnA6wCRuBjnorIG8BAoJmIbAYeBCIBVPVFVV0pIrOBpXgu2U9S1ZKuar8HXnNarK8FbB70Sir5/U+Zk8mW7Dzi42IZfWYHpn6/nuEvzyN1dF96t7eRDU35fOp+JiLpzn0wV1iXldDxyo/reeD95Vzfvx2PXH6q9xjhppqSkpJYvHixz11W3GC57Jus7DyGv/wTO3MOMWVkH/qdeILbIZlaVuPdz4AvRORKsb+6phpmL9vGgx8s54KuLXn4MiviNe28884DiLM8DX4JcbG8desAWjWOYcTU+Xy3epfbIZkA5mshvxV4GzgkIvtFJEdE9vsxLhNiFm7Yy51vptOjTRzPXJtEeJjVmpr20ksvAXTC8jQktGwUw1u3DqDDCfUZPW0BX63a4XZIJkD5NB+5qjb0dyAm9MxKzzpy708EmtaLZPKIZGKjbMAXf8jJyanU5TgT+Jo1iOaNW/pzw5R5jHk1jWev68Ugm0jIHMPnCRVEpImI9BWRs0oe/gzMBLdZ6VmMn5lBVnYeChQr5Bwq4lu7ROhv4ZanoaVJ/Sheu7k/3eIbc/tri/hwyRa3QzIBxtchWm/G0390DvCw8+9D/gvLBLuUOZnkFRw9FsmhwmIbscqPJk2aBJ6eJZanIaZxbCTTb+5H73ZNuPPNdGYu2ux2SCaA+HpGfifQB9igqucASYB1cjRlshGrat/TTz8NsBLL05DUIDqC1NF96H/iCdzz9hLenL/R7ZBMgPC1kOeraj6AiESr6io83/yNOY6qlnkf3IZh9Z+YmBj4dVQ3y9MQVC8qgikj+3DWSc25b2YGr/y43u2QTADwtZBvdqYznAV8JiLvAxv8F5YJZs99tYaDh4uIOKZlemxkOONsnnG/adOmDXiGaLU8DWExkeFMvLE3F3RtyQPvL2fSt2vdDsm4zNdW61c4Tx8Ska+AxsBsv0VlgtY7CzfzxKc/MzQpgf87qRlPfPrzkRGrbJ5l/3rvvfcQkSJVtTwNcdER4Tw/vBd3vbmYf3y8kvyCIu449yS3wzIu8amQi0g7r5frnH9bAXaTxhzx7eqd3PfuUs7ofAKPXXkaURFhXNGrjdth1RkbN24EiHLy1fI0xEWGh/H0tT2JigjjiU9/5lBhMXdfcLINtFQH+VTIgY/x3HsTPOOudwQygW5l7SAiU4DfADtU9VRn2UPALXjmMQb4i6p+UqXITUBZvmUft01fROcWDXjh+t5ERfjcs9HUkEsuuQSgM5589SlPTXCLCA/jiat7EBUexn+/XMPhwmLuu6iLFfM6xtdL6929X4tIL+D2CnZLBZ7FMwOTt3+r6hO+BmgCX1Z2HqOmLqBhTASpo/rSKCbS7ZDqpIyMDERkRcmAMD7mqQly4WHChKHdiYoI46Vv1nKosJgHL+1qxbwO8fWM/CiqukhE+lWwzTci0qEqxzfBY9/BAkZOmU9eQRHvjD2dVo1j3A7JOHzJUxMawsKEv1/ejaiIMCZ/t47MbfvZsOcgW7PzrX1KHeDrPfK7vV6GAb2Aqg4vdIeI3AikAfeo6t4y3nMMMAagXbt2pW1iXHaosIhbXk1jw+6DTBvdl8RWNpKvm5z5yFs6+VrdPDVBRkT42yWnsHF3Lp+t/HVc9qzsPMbPzACwYh6ifL2R2dDrEY3nHtzlVXi/F/BM6tAT2Ao8WdaGqjpRVZNVNbl58+ZVeCvjT8XFyt0zljB/3R5Srj6NAZ1smkW35eTkgCenq5unJkiJCCu25hy3PK+gyEZVDGG+3iN/uCbeTFW3lzwXkZeBj2riuKb2TfjfSj5eupXxF3Xh8p72LT8QPPjggzz00ENbaypfTXCyURXrHl8vrX+IM2JUaVT1Mh+P01pVtzovrwCW+bKfCSxTvlvHy9+uY8SA9ow560S3wzGOSy+9FKCziHxQ2npf89QEt/i4WLJKKdrWfiV0+drYbS2e/qjTndfDgO14RpAqlYi8AQwEmonIZuBBYKCI9MTzpWA9nnnOTRD5X8ZWHvl4BYO6teSBS7tZy9gAcuKJJwIUAy87iyrMUxN6xg1KZPzMjOMmLYoIE/bmHqZJ/SiXIjP+4mshP+OYOY4/FJE0Vf1jWTuo6rBSFk+uVHQmoKSt38Odby0mqW0cT1+bRHiYFfFA8v333wOsVdUPnUUV5qkJPSUN2lLmZB4ZVfGi7q145ccN/Hbij0y/qR8tGtnZeSjxtZDXF5ETVXUtgIh0BOr7LywTaNbsOMDNr6TRJi6WSSP6EBNZ+qQoxj25ubkAR063LE/rriFJCce1UD+3SwtumZbG1S95innbpvVcis7UNF9brf8RmCsic0Xka+ArPFObmjpgR04+I6fOJyJMSB3Vl6Z2aS4g/fvf/wZItDw1pTm9UzOm39yP7IMFXP3ij6zZccDtkEwN8amQq+ps4CQ8fxT+ACSq6qf+DMwEhtxDhYxOXcDuA4eZMrIP7U6wb/GBavDgweBpQGp5akqV1K4Jb47pT2Gxcs1LP7Isa5/bIZkaUG4hF5E+ItIKQFUPAT2AvwMpItK0FuIzLiooKub21xaxcmsOzw/vxWlt4twOyZRiwYIFbNu2reSlYnlqynFK60a8PXYAsZHhDJv4E2nr97gdkqmmis7IXwIOA4jIWcBjeMZO3wdM9G9oxk2qyt/eW8bXP+/kH0NO5ZwuLdwOyZTh1ltvJSrqyO2OBliemgp0bFafGWMH0LxhNDdMns83P++seCcTsCoq5OGqWvJ17bfARFV9V1XvxzPLkglRz3yxhrfSNvGHczszrK8NkRvIioqKaNr0yIl3UyxPjQ8S4mJ569YBdGhWn5unpTF72baKdzIBqaJW6+EiEqGqhcB5OGOf+7ivCSKz0rOOdFdpXC+S7IMFXNmrDX+84GS3QzMVKCoqorCwkIiICPAMz/ql12rLU1Om5g2jefOW/oxMnc/vXl/Ev648jSt7t3E7LFNJFSX5G8DXIrILyAO+BRCRzngu25kQMCs966gBJLIPFhAmcHqnpjbgSxAYNmwYZ599Ns2aNQPPPXLLU+OzxvUimX5TP8a8msY9by8h93AhNw7o4HZYphLKvbSuqo8C9+CZW/xMVS0ZpjUM+L1/QzO1JWVO5nGjQBUrPPXZapciMpXx17/+lSeffJKRI0cCrLI8NZVVPzqCySP6cP4pLXng/eU899Uat0MylVDhZTdV/cn7tYiMUVVrQBNCbJKF4Ne/f/+Sp8VgeWoqLyYynBeu78Wf3l5CypxMcvIL+fPgRLsqFwR8HRDG29gaj8K4qqzJFOLjYms5ElODLE9NpUWGh/Hva3oyvF87Xvz6F+5/fxnFxWXOl2UCRFUawtjXsxCSX1BETMTx3+diI8MZNyjRhYhMDbE8NVUSFib8Y8ipNIiJ4KWv15J7qIiUq04jIrwq532mNlSlkF9a41EYVxQXK398azHrdh9kxID2fL5yx5FJFsYNSjxurGYTVCxPTZWJCPcN7kKjmEhS5mSyensOe3IPs3Vfvv19CEC+zkceB9wIdAAiSu6ZqOof/BaZ8bt/fLyS/y3bxt8uOYWb/+9EHr7c7YhMdWRnZwO0EJGnsDw11SQi/O6czvyy8wAzF2UdWZ6Vncf4mRkAVswDhK/XSj7BU8QzgIVeDxOkJn27linfr2P0GR25+f9OdDscUwMuvvhi8Mx+5nOeisgUEdkhIsvK2WagiCwWkeXOZCze68JFJF1EPqr+JzCBaN7a44dwzSsoImVOpgvRmNL4emk9RlXv9mskptZ8tHQL//h4JRd3b8XfLjnF7XBMDcnPzwfYrKpTK7FbKvAsniFdj+NcjXseGKyqG0Xk2LF67wRWAo0qHbAJCtarJfD5ekb+qojcIiKtRaRpycOvkRm/mLd2N3e/tYQ+HZrw1DU9CQuzNlGh4oYbbgBoVpk8VdVvgPJmzbgOmKmqG53td5SsEJE2wCXApGoHbwJWWb1XWjSKruVITFl8LeSHgRTgR369XJfmr6CMf6zensMtr6TRtmksL9+YTExkuNshmRrkTJzShprN05OBJs4c5wtF5Eavdf8B7sXpu14WERkjImkikrZzp03OEWzGDUoktpS/FXmHC1m9PceFiMyxfC3k9wCdVbWDqnZ0HnZjNYhs35/PyKkLiI4MJ3VUX+LqRVW8kwkqTz75JMCyGs7TCKA3njPvQcD9InKyiPwG2KGqFbaVUdWJqpqsqsnNmzevZjimtg1JSmDC0O4kxMUieCZbuXdwItGREVzz0o8s2ZTtdoh1nq/3yNcAB/0ZiPGfA4cKGTV1AXsPHmbGrQNo27Se2yEZP+jcuTPr1q0r9+y4CjYDu1U1F8gVkW/wzHfeC7hMRC4GYoBGIjJdVa+v4fc3AWBIUsJxLdQv6d6a6yfP47qXf2LSiD4M6HSCS9EZX8/Ic4HFIvKSiDxT8vBnYKZmFBQVc9v0hWRuz+H54b04NaGx2yEZP6lfvz5A1xrO0/eBM0UkQkTqAf2Alao6XlXbqGoH4FrgSyvidUv7E+rz9q2nEx8Xy4ip8/lsxXa3Q6qzfC3ks4BHgR+w7mdBQ1UZPzODb1fvYsIV3RmYeGyDYxNKhgwZArCVSuSpiLyB5556oohsFpGbRGSsiIwFUNWVwGxgKTAfmKSqZXZVM3VLq8YxzLh1AKe0bsTY6QuZuWiz2yHVSfLrREmBKzk5WdPSrG1dZT312c8888Vq7jr/JO463+YVrwtEZKGqJrsdR1ksl0PTgUOFjHkljR9+2c1Dl3Zl5Bkd3Q4p6FUml30d2W0dnnmOj2IN3gLXm/M38swXq7kmuQ13nneS2+GYWtCxY0eA7iKy1nu55anxtwbREUwZ2Yc/vJHOQx+uYH9+Ib8/t7PNnFZLfG3s5v2tIAa4GrB+5AHqq1U7+OusZZx9cnMevaK7JVMdkZaWRrNmzVYA52N5ampZTGQ4zw/vxb3vLuWpz34m+2ABf7vkFBurohb4VMhVdfcxi/4jIguBB2o+JFMdSzdnc/trizildUOeH96LSJuxqM444YQTAIq88tXy1NSqiPAwnriqB41iIpny/Tr25xfw2NDuNnOan/l6ab2X18swPGfoVZk5zfjRxt0HGZ26gBMaRDFlZB/qR9uPqC5ZtGgRQD0nXy1PjSvCwoQHL+1KXL1I/vP5anLyC3hmWBLRETYAlb/4muRPej0vBNYD15S3g4hMAUoGjTjVWdYUeAvPBCzrgWtUdW+lIjal2pN7mBFT51NQpLw5pi8tGsa4HZKpZffccw94RnZ7Eh/z1Bh/EBHuOv9kGsVE8vePVjA6dQETb0i2kws/8fXS+jlVOHYqx0/GcB/whao+JiL3Oa//XIVjGy/5BUXcPG0BWdl5vHZzPzq3aOB2SMYFX331FSLycxXz1ZgaN/rMjjSOjeTed5cyfNI8Ukf1sVEl/aDcGxcicqmItPd6/YCILBGRD0Sk3P4FZUzGcDkwzXk+DRhShZiNl6Ji5c4300nflM3Tv+1Jnw7Wtqmu+fDDD9mwYcOR15XJU2P87crebXhheC9WbNnPb1/6iR37890OKeRU1ALhUWAngDO28vXAaOAD4MUqvF9LVd3qPN8GtKzCMQwwKz2LMx77gk5/+YQ5y7czpEc8F3Vv7XZYxgV//etf8RrDvDHVz1NjatSF3VqROqoPm/ce5KoXf2TjbhvxuyZVVMhVVUv+x4cCk1V1oapOAqo1+4F6RqIpczQamzGpbLPSsxg/M4Os7F+/2c5evp1Z6VkuRmXcIiLUq3dk/Pwm1GCeGlNTTu/cjNdu6c/+/AKuevEHXpi7hjMe+5KO933MGY99aX+/qqGiQi4i0kBEwoDzgC+81lWlNdV2EWntHLg1sKOsDW3GpLKlzMkkr6DoqGV5BUWkzMl0KSLjJlXlwIEDFBcXAzSk+nlqjF/0bBvHjFsHkF9QyOOzM8nKzkOBrOw8xs/MsGJeRRUV8v8Ai/HMabxSVdMARCQJz5jOlfUBMMJ5PgLPhAymkrKy80pdvqWM5Sa03XXXXfTs2ZPk5GSA/BrIU2P85uSWDYmNOr6dtZ2MVF25rdZVdYqIzAFaAEu8Vm0DRpW3rzMZw0CgmYhsBh4EHgNmiMhNwAasa0yl/bw9B6H0exLxcbG1HY4JAKNHj2bQoEHs2LGDXr16rfZaVWGeGuOGHfsPlbrcTkaqptxCLiIdVHU9cNT1jpIGa+IZ+zNBVY+b8kZVh5Vx2POqFqrZti+fkVPm0yA6nMNFyqHCX6eejo0MZ9ygRBejM25Zv349HTp0ICHh6PmifclTY9wQHxdb6pVFOxmpmor6kac498ffxzMd4k4899w6A+fgKcoPAvYHws9y8gsYOXU++/IKmDF2AKu3HyBlTiZbsvOIj4tl3KBEhiQlVHwgE3LGjRtHcXExl19+OUCMiLTA8tQEsHGDEhk/M+O4tj7J7Zu4FFFwq+jS+tUi0hUYjqc7S2sgD1gJfAw8qqrWKdDPDhcWc9v0RazZcYApI/vQLb4x3eIbW+E2ALz99tusWLGC1157DaA98C2WpyaAlfztKjkZad04hhaNonl/yRbaNI3lTxcm2mRPlWDzkQc4VeWeGUuYmZ7FE1f34KrebdwOyQQwm4/cBKuiYuVvs5bxxvyNXNevHY9cfirhdXjmNH/MRz60lMX7gAxVLbMLmam+Jz/9mZnpWdxzwclWxE25Zs6cCRB3TL5anpqgEB4m/POKU2lSL5Ln5/7C/rwCnrqmJ1ERNnNaRXwdwf4mYADwlfN6IJ575h1F5O+q+qofYqvzXpu3gWe/WsOwvm2549zObodjAtzkyZPBMyHRcGfRQCxPTRAREe4d3IW4epH885NV5OQX8sL1vahXSnc18ytfv+pEAKeo6pWqeiXQFU8PqH7YpCd+8cXK7dw/axnnJDbnkctPtftFpkKFhYUAyyxPTbAbc1YnHr+yO9+u3skNk+ez72CB2yEFNF8LeVtV3e71eoezbA9g/8M1bPGmbO54PZ1TExrz7HW9iAi3S0umYps2bQLP9KUlLE9N0Pptn3Y8d10vMjbv47cTf2RHjrXXLIuvFWKuiHwkIiNEZASeEdrmikh9INt/4dU963flclPqApo1jGLyiD42f6/x2cCBAwE6W56aUHFR99ZMGdmHjXsOcvWLP7Jpj022UhpfC/nvgKlAT+cxDfidquba3Mc1Z/eBQ4ycOp9iVaaN6kvzhtFuh2SCyHPPPQewC8tTE0LOPKkZr93cj+yDBVz5wg9kbstxO6SA41Mhd2Yq+w74Es+EDN9oMPRbCyJ5h4u4aVoaW/flM2lEMic2b+B2SCbIOO0oDmB5akJMUrsmvD12ACJwzUs/smjjXrdDCig+FXIRuQaYD1yFZ3z0eSJylT8Dq0uKipXfv5HOks3ZPH1tEr3bN3U7JBOEZsyYAXAKlqcmBJ3csiHvjD2duHqRXD9pHt+utumtS/h6af2vQB9VHaGqNwJ9gfv9F1bdoao8+MEyPl+5nYcu7cbgU1u5HZIJUo8++ih4Zim0PDUhqW3Terw9dgDtmtZjdOoCPsmwyf3A90IedsyAErsrsa8px4tfr2X6Txu59awTGXF6B7fDMUHMmY/cu9W65akJOS0axvDWmAGc1iaOO15fxJvzN7odkut8TfLZIjJHREaKyEg84zd/4r+w6oZZ6Vk8PnsVl/WI58+Du7gdjglygwcPBjjJ8tSEusb1Inn1pr7830nNuW9mBi9+/YvbIbnK57HWReRK4Azn5beq+p7fojpGKI7P/MOaXYyYOp/e7ZswbXRfoiPC3Q7JhAAR+QVPtzOo5Tz1RSjmsnHP4cJi7p6xmI+WbuW8Li1YtW0/W7LzQ2JGyBofax1AVd8F3q1yVOaIVdv2c+urC+nYrD4v3ZBsRdzUpGxVvdvtIIypDVERYTx9bRJ7cg/xxapf7/5mZecxfmYGQFAXc1+VW8hFJAfPEI/HrcLTK62RX6IKYVv35TFyygLqRYeTOqovjWMj3Q7JBLmGDRt6D+GbJCL7neeWpybkhYcJG3YfP1BMXkERKXMyrZCrasPaCiSUzUrPOjLvbniYEC4w644ziY+LdTs0EwJycn4dIENE0gN5GlNj/GFLdunDt27JzqvlSNxhLVr9bFZ6FuNnZpCVnYcChcWKIjY6kTHG1JCyTopaNY6p5UjcYYXcz1LmZJJXUHTUssNFxaTMyXQpImOMCS3jBiUSG3l8WyNVZfv+0J9sxQq5n5V1aaeuXPIxgU1EpojIDhFZVs42A0VksYgsF5GvnWVtReQrEVnhLL+z9qI25mhDkhKYMLQ7CXGxCJAQF8vtAzuRk1/IlS/8wPpduW6H6Fc2tZafNYqNZF/e8TNI2v1xEyBSgWeBV0pbKSJxwPPAYFXdKCItnFWFwD2qukhEGgILReQzVV1RG0Ebc6whSQnHNWwb1K0VI6fO56oXf+SV0X3pGh+a7T7tjNyPPl2+jX15BYTJ0ctjI8MZNyjRnaCM8aKq3wB7ytnkOmCmqm50tt/h/LtVVRc5z3OAlUDoNw82QaVH2zjeHns6keHCbyf+yIL15f2qBy8r5H6yaONe/vBmOj3axvHYMZd8JgztXie6RJiQcDLQRETmishCEbnx2A1EpAOQBMwr7QAiMkZE0kQkbedOm+jC1K7OLRrwzm2n07xBNDdMnsdXq3ZUvFOQsUvrfrBuVy43T0ujZaMYJo9IplmDaK7p087tsIypigigN3AeEAv8KCI/qerPACLSAM9AUXep6v7SDqCqE4GJ4BnZrVaiNsZLQlwsb48dwIip87nllTSeuLpHSJ1M2Rl5Ddt14BAjpswHYNqovjRrEO1yRMZUy2Zgjqrmquou4BugB4CIROIp4q+p6kwXYzSmQic0iOaNW/rTu30T7nprMdN+WO92SDXGCnkNOni4kJtSF7AjJ5/JI5Lp0Ky+2yEZU13vA2eKSISI1AP6ASvFM5TcZDzTpj7laoTG+KhhTCTTRvfl/FNa8uAHy3n689X4Ot9IILNCXkMKi4r5/evpZGTt47/DepHUronbIRlTIRF5A/gRSBSRzSJyk4iMFZGxAKq6EpgNLAXmA5NUdRmeCZRuAM51uqYtFpGLXfoYxvgsJjKcF6/vxZW92vDvz3/m4Q9XUFwc3MXclXvkIrIeyAGKgMJgH1JSVbn//eV8sWoH/xhyKhd0bel2SMb4RFWH+bBNCpByzLLv8IzlbkzQiQgPI+Wq04irF8nk79axL6+Af111GpHhwXlu62Zjt3Oce25B7/m5v/DG/I3cPrAT1/dv73Y4xhhjKhAWJvztklNoUi+SJz79mf15BTw3vBcxpYwQF+iC8+tHAHl34WZS5mRyRVKC9Q03xpggIiLcce5JPDLkVL7M3MGNk+ezP//4AbwCnVuFXIFPnX6pY0rbIBj6nn67eid/fncpZ3Q+gcevPM17KkljjDFB4ob+7Xn62iQWbdzLtS/9xM6cQ26HVCluFfIzVbUXcBHwOxE569gNVHWiqiaranLz5s1rP8IKrNiyn9umL6Jziwa8cH1voiLs4oYxxgSry3rEM2lEMut25XL1iz+wac/xc5wHKleqj6pmOf/uAN4D+roRR1VlZecxKnU+DWMimDqqD41iIt0OyRhjTDUNTGzB9Jv7sif3MFe/+CMvzl3DGY99Scf7PuaMx75kVnqW2yGWqtYLuYjUdyZZQETqAxcCZc68FGj2HSxg5JT5HDxcROqovrRubJOfGGNMqOjdvilv3TqA3EMFPDY7k6zsPBTPCdz4mRkBWczdOCNvCXwnIkvw9Ev9WFVnuxBHpR0qLGLMq2ms353LSzf0JrFVQ7dDMsYYU8NOad2IetHHd+rKKygiZU6mCxGVr9a7n6nqWpwhHoNJcbHyp7eXMm/dHp6+tiend2rmdkjGGGP8ZMf+0hu8bcnOq+VIKmYttHz0+OxVfLhkC+Mv6sLlPUNnsH1jjDHHi48r/bZpWcvdZIXcB6nfr+Olb9YyYkB7xpx1otvhGGOM8bNxgxKJLWVwmKR2jQNufHYr5BWYvWwrD3+0ggu7tuSBS7tZX3FjjKkDhiQlMGFodxLiYhEgPi6GXu3i+GjpNib8b1VAFXObj7wUs9KzSJmTyRantWL7prE8MyyJ8DAr4sYYU1cMSUo4at7y4mLloQ+XM/GbtezNPcyEod2JCIDx2d2PIMDMSs9i/MyMI10OALbnHGL2sm2uxmWMMcZdYWHCw5d1487zTuLthZu5/bVF5BcUuR2WFfJjpczJJO+YH0x+QXFAdjkwxhhTu0SEP15wMg9e2pVPV2xn1NQF5Lg8PrsVci+b9hwkq4yuBYHY5cAYY4w7Rp3Rkf/8tifz1+/hupfnsfuAe+OzWyEH8g4X8dSnmZz/1NdlTrAciF0OjDHGuGdIUgIv39ibn7fncPVLP5Z5IuhvdbqQqyofLd3CeU/O5Zkv1zCoWysevKzrcV0OYiPDbYpSY4wxxzm3S0tevakfO/cf4qoXfmDNjgO1HkOdLeQrt+7n2ok/ccfr6cTVi2LGrQN4ZlgSI0/veFSXg4S4WCYM7X5Uy0VjjDGmRN+OTXnz1v4UFClXv/gDSzdn1+r717nuZ3tzD/PUZz/z2rwNNHyRkjoAAAjZSURBVI6N5NErTuXaPu2O6lp2bJcDY4wxpjzd4hvzztgBXD95HsMm/sTLNyZzeufaGcq7zpyRFxUrr/60gXOenMvr8zdy44AOfPWngQzv1976hxtjjKm2Ds3q8+5tp5PQJJaRUxcwe9nWWnnfOlHI563dzW/++x33z1rGKa0a8fEfzuShy7oRVy/K7dCMMcaEkJaNYphx6wC6JTTi9tcW8daCjX5/z5C+tL4lO49/frKSj5ZuJSEulueH9+KiU1vZMKvGGGP8Jq5eFK/d3I+x0xfx53czyD5YwK1nd/Lb+4VkIc8vKGLiN2t5fu4aVOGu80/i1rM6ERt1/AD4xhhjTE2rFxXBpBuTuXvGYib8bxV7Dh7mvsFd/HIiGfSF3Htc9Pi4GC7s1orPVmxn8948Lu7eir9cfAptmtRzO0xjjDF1TFRE2P+3d7cxdtRVHMe/P3ZX3YK2aE0FWl2itaagUG0EwRhCNaAQSmIMxYcQYyQahWpMpbzyDTHEGGOr9aFWSg0FXtQKxBe1pGAxWhWLpY8aSK3Llq3dqottbUutxxfzX7ndvXfb7c7dmen8Pslm5/53OnNubs89d+b+5wxLFsxhcncXP9y4m2d7B+n957/pHzzKhVO6WXTdrFwmVle6kA/1RR9qqbp38Cgrf72HN73u1Tz42Su46q0TM2PQzMysmY5zxD03X8rAwaOs37n//+N7B49w99ptAOMu5pWe7NasLzpkje1dxM3MrAwksePFgyPGjxw/kct9PCpdyFv1P+8fPDrBkZiZmbXWql7lcR+PShfyVv3P3RfdzMzKpJ31qtKFfNF1s9wX3czMSq+d9arSk92GJgi8Mms9v1mAZmZmeWlnvap0IQf3RTczs2poV72q9Kl1MzOzunMhNzMzqzAXcrMak3SfpP2Sto+yzjWStkjaIWljw/j1kv4s6XlJiycmYjMbzoXcrN7uB65v9UdJU4DvATdFxCXAx9J4B7AM+DAwG7hV0uy2R2tmI7iQm9VYRDwF/GOUVT4OrI2I3rT+UI/J9wLPR8TuiHgZeBiY39ZgzaypSsxa37x58wFJfy1g11OBAwXsdywcYz7OlhjfkvM+3w50Sfol8FpgSUT8BLgIeKFhvT7gimYbkHQ7cHt6eEjS+HtSjt3Z8voWzTHmI9dcrkQhj4g3FrFfSX+IiLlF7Pt0OcZ8OMaWOoH3APOAbmCTpN+OZQMRsRxY3obYTptf33w4xnzkHWMlCrmZFaYP+HtEHAYOS3oKuCyNz2hYbzqwt4D4zGrP35Gb2WgeBd4vqVPSJLLT57uAp4GZki6W9CpgAfBYgXGa1ZaPyEdX6OnA0+QY81HLGCU9BFwDTJXUB3wN6AKIiB9ExC5J64CtwH+BFRGxPf3bLwK/ADqA+yJiR97x5aiWr28bOMZ85BqjIiLP7ZmZmdkE8ql1MzOzCnMhNzMzqzAX8mEkzZD0pKSdqSXlwqJjakVSh6Q/Svp50bE0I2mKpDWS/iRpl6T3FR3TcJK+nF7n7ZIekvSaEsQ0om2qpNdLelzSc+n3+UXGWBVVyeey5zI4n8cRU9vz2YV8pP8AX4mI2cCVwBdK3HpyIdkM4rJaAqyLiHeQXbJUqlglXQTcCcyNiEvJJm0tKDYqoHnb1MXAhoiYCWxIj+3UqpLPZc9lcD6fqftpcz67kA8TEf0R8UxaPkj2n7V0NzyXNB24AVhRdCzNSJoMfAD4MUBEvBwRg8VG1VQn0C2pE5gEvFhwPK3aps4HVqXlVcDNExpURVUhn8uey+B8Ho+JyGcX8lFI6gHmAL8rNpKmvg18leySoDK6GBgAVqZThisknVt0UI0iYi/wTaAX6Adeioj1xUbV0rSI6E/L+4BpRQZTRSXO57LnMjif85ZrPruQtyDpPOCnwJci4l9Fx9NI0o3A/ojYXHQso+gE3g18PyLmAIcp2eng9L3UfLI3qQuBcyV9stioTi2ya0Z93egYlDWfK5LL4Hxumzzy2YW8CUldZEm/OiLWFh1PE1cDN0naQ3bXqWslPVBsSCP0AX0RMXT0s4bsjaBMPgj8JSIGIuI4sBa4quCYWvmbpAsA0u/9p1jfkpLncxVyGZzPecs1n13Ih5Eksu+BdkXEt4qOp5mIuDsipkdED9lkjiciolSfPCNiH/CCpFlpaB6ws8CQmukFrpQ0Kb3u8yjZBJ4GjwG3peXbyFqn2imUPZ+rkMvgfG6DXPPZhXykq4FPkX0y3pJ+PlJ0UBV1B7Ba0lbgcuDrBcdzknR0sQZ4BthGlg+Ft3dMbVM3AbMk9Un6DHAv8CFJz5EdedxbZIwV4nzOj/P5DExEPrtFq5mZWYX5iNzMzKzCXMjNzMwqzIXczMyswlzIzczMKsyF3MzMrMJcyGtK0omGy3G2SMqtS5OknsY7/ZhZ+ziXrbPoAKwwRyLi8qKDMLNxcy7XnI/I7SSS9kj6hqRtkn4v6W1pvEfSE5K2Stog6c1pfJqkn0l6Nv0MtUTskPSjdG/g9ZK60/p3pntDb5X0cEFP0+ys51yuDxfy+uoedjruloa/vRQR7wS+S3ZnJoDvAKsi4l3AamBpGl8KbIyIy8h6L+9I4zOBZRFxCTAIfDSNLwbmpO18rl1PzqxGnMs1585uNSXpUESc12R8D3BtROxON5vYFxFvkHQAuCAijqfx/oiYKmkAmB4Rxxq20QM8HhEz0+O7gK6IuEfSOuAQ8AjwSEQcavNTNTurOZfNR+TWTLRYHotjDcsneGU+xg3AMrJP/E9L8jwNs/ZxLteAC7k1c0vD701p+Tdkd2cC+ATwq7S8Afg8gKQOSZNbbVTSOcCMiHgSuAuYDIw4kjCz3DiXa8CfoOqrW9KWhsfrImLospXz0x2OjgG3prE7gJWSFgEDwKfT+EJgebqjzwmyN4L+FvvsAB5IbxAClkbEYG7PyKyenMs15+/I7STpe7W5EXGg6FjM7Mw5l+vDp9bNzMwqzEfkZmZmFeYjcjMzswpzITczM6swF3IzM7MKcyE3MzOrMBdyMzOzCvsf1TwvPHfsZuEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f515054bc50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(nrows=1, ncols=2, figsize=(8,4))\n",
    "ada1 = AdalineGD(n_iter=10, eta=0.01).fit(X, y)\n",
    "ax[0].plot(range(1, len(ada1.cost_) + 1),\n",
    "          np.log10(ada1.cost_), marker='o')\n",
    "ax[0].set_xlabel('Epochs')\n",
    "ax[0].set_ylabel('log(Sum-Squared-Error)')\n",
    "ax[0].set_title('Adaline - Learning rate 0.01')\n",
    "\n",
    "ada2 = AdalineGD(n_iter=10, eta=0.0001).fit(X, y)\n",
    "ax[1].plot(range(1, len(ada2.cost_) + 1),\n",
    "          np.log10(ada2.cost_), marker='o')\n",
    "ax[1].set_xlabel('Epochs')\n",
    "ax[1].set_ylabel('log(Sum-Squared-Error)')\n",
    "ax[1].set_title('Adaline - Learning rate 0.0001')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "\n",
    "分析上面两幅图各自的问题，左图根本不是在最小化损失函数，反而在每一轮迭代过程中，损失函数值不断在增大！这说明取值过大的学习率不但对算法毫无益处反而危害大大滴。右图虽然能够在每一轮迭代过程中一直在减小损失函数的值，但是减小的幅度太小了，估计至少上百轮迭代才能收敛，而这个时间我们是耗不起的，所以学习率值过小就会导致算法收敛的时间巨长，使得算法根本不能应用于实际问题。\n",
    "\n",
    "下面左图展示了权重再更新过程中如何得到损失函数$J(w)$最小值的。右图展示了学习率过大时权重更新，每次都跳过了最小损失函数对应的权重值。\n",
    "\n",
    "![](https://ooo.0o0.ooo/2016/06/15/57620d14cfd33.png)\n",
    "\n",
    "许多机器学习算法都要求先对特征进行某种缩放操作，比如标准化\\(standardization\\)和归一化\\(normalization\\)。而缩放后的特征通常更有助于算法收敛，实际上，对特征缩放后在运用梯度下降算法往往会有更好的学习效果。\n",
    "\n",
    "特征标准化的计算很简单，比如要对第$j$维度特征进行标准化，只需要计算所有训练集样本中第$j$维度的平均值$u_{j}$和标准差$\\sigma_{j}$即可,然后套公式：\n",
    "\n",
    "$$x_{j}^{'}=frac{x_{j}-u_{j}}{\\sigma_{j}}$$\n",
    "\n",
    "\n",
    "标准化后的特征 均值为0，标准差为1。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在Numpy中，调用mean和std方法很容易对特征进行标准化:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_std = np.copy(X)\n",
    "X_std[:, 0] = (X[:, 0]-X[:, 0].mean()) / X[:, 0].std()\n",
    "X_std[:, 1] = (X[:, 1]-X[:, 1].mean()) / X[:, 1].std()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "标准化后，我们用Adaline算法来训练模型，看看如何收敛的\\(学习率为0.01\\)："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<__main__.AdalineGD at 0x7f5150574490>"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ada = AdalineGD(n_iter=15, eta=0.01)\n",
    "ada.fit(X_std, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "我们将决策界和算法学习情况可视化出来:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEWCAYAAABmE+CbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3Xm8HFWZ//HPN2EJmAthCcgSEmBYjKgIYRMJIKjguIEyoo78EBTFDRdGB5iXojJEcdBRUBCJIIKiDKKIyCoQQAkhyL4EZAv7DgkxQJLn98c5Tfpe7u2um65e7/f9evXrdi1d9VQH6ul66pxTigjMzMxGtTsAMzPrDE4IZmYGOCGYmVnmhGBmZoATgpmZZU4IZmYGOCHYMpK0v6SrCq57pKTT8/sNJM2XNLq5EbaPpFMlHZXf7yTpznbHZFaEE4L1I+lySc9IWrEZ24+IByJibEQsbsb265E0RdJ5+RiflXSbpP+WtFoz9hcRV0bEZmVsS9J9knavsXwXSUtywp0v6UFJv5W0TRn7b4b839sn2h2HJU4I9gpJk4CdgADe29ZgmkDSW4DLgauBzSNiHLAHsAh40xCfWa5lAZbj4YgYC/QB2wN3AFdK2q29YVk3cEKwavsB1wCnAv+veoGkNSSdK+l5SdcCGw9Y/kNJc/Py2ZJ2GmwHkiZJisqJNv9C/LakqyXNk3SRpDWr1t9e0l/zr/kbJe3SwPEdA5wSEdMi4jF45YrlGxFxed7f/jmWH0h6CjhS0saS/iLpKUlPSjpD0riqGN8s6foc/2+AMVXLdpH0YNX0upLOlvSEpHslfaFq2ZH5F/1peVu3SpqSl/0S2AD4Y/71/9VaBxrJgxHxdeBk4LtV+9lc0sWSnpZ0p6R/q1r2rnzVNE/SQ5IOrVr2Pkk35H/jf0jaI89fVdJ0SY/kzxxVKQlWSouS/idfld0rac+87L9JP0COz8d0fNF/SGuSiPDLLyIC4G7gM8DWwMvA2lXLzgR+C7wG2AJ4CLiqavm/A2sAywFfAR4FxuRlRwKn5/eTSFcgy+Xpy4F/AJsCK+Xp7+Rl6wFPAe8i/Xh5e54evwzH9hpgMbBLnfX2J10xfD4fy0rAv+R9rwiMB2YA/5vXXwG4H/gSsDzwwfzdHZWX7wI8mN+PAmYDX8+f2wi4B3hn1fe0MB/vaGAacE1VbPcBu9eI/ZV9DZj/NmBJ/g5eA8wFPp6P783Ak8DkvO4jwE75/WrAVvn9tsBz+XsYlf9tNs/LzgF+mre9FnAt8Kmq7/Nl4JP5mA4GHgZU9e//iXb/t+9XevkKwQCQ9FZgIvDbiJhNOkl/JC8bDXwA+HpEvBARtwC/qP58RJweEU9FxKKIOJZ08ixaOz8lIuZExD9JSWfLPP/fgfMj4vyIWBIRFwPXkU6Yw7Ua6UT2aNUxH5OvPF6Q9F9V6z4cEcflY/lnRNwdERdHxIsR8QTwfWDnvO72pETwvxHxckT8HzBriBi2ISWzb0XESxFxD/AzYN+qda7Kx7sY+CVDlLKG6WFAwDjg3cB9EXFKPr6/A2cD++R1XwYmS1olIp6JiOvz/AOBn+fvYUlEPBQRd0ham/Tv8cX838bjwA8GHNP9EfGzfEy/ANYB1i7huKxkTghW8f+AiyLiyTz9K5aWjcaTfk3OrVr//uoPSzpU0u2SnpP0LLAqsCbFPFr1fgEwNr+fCOyTT9rP5u2+lXRC6UfSR6tupv55kH08Q/qV/MpnI+Krke4jnJOPr6L6OJG0tqQzcznkeeD0qmNbF3goIqpHiez33VSZCKw74HgOp//JceB3MaaE+xjrka7Kns0xbDcgho8Cr83rfoB0gr9f0hWSdsjzJ5B+JAx2TMsDj1Rt76ekK4VXHVNELMhvx2Idp9tumFkTSFoJ+DdgtKTK/7wrAuMkvQm4hVRGmUC6SQmpnl35/E7AV4HdgFsjYomkZ0i/ShsxF/hlRHyy3ooRcQZwRo3lL0iaCewNXFZvcwOmj87z3hART0t6P1Cpdz8CrCdJVUlhAwY/ec4F7o2ITersv2hcRe0FXJ+/g7nAFRHx9kF3EDELeJ+k5YHPka7YJpBi33iQj8wFXgTWjIhFyxCbh1vuIL5CMID3k+rrk0nlmi2B1wFXAvvlS/3fkW6wrixpMv1vOveREsYTwHKSvg6sUkJcpwPvkfROSaMljck3addfxu19FThA0n9KWgsgb2vDOp/rA+YDz0laD/iPqmV/Ix37FyQtL2lvUr19MNcC8yR9TdJK+Zi2UPFmoY+R7jvUpWQ9Sd8APkG6EgE4D9hU0sdyvMtL2kbS6yStkK+0Vo2Il4HnSVdVANOBj0vaTdKovO3NI+IR4CLgWEmr5GUbS9p5YEyNHpM1nxOCQTq5nxKpxc2jlRfpV/BHc8nic6TL/EdJrZBOqfr8hcAFwBxSuWQhA8ouyyIi5gLvI53Mnsjb/A+W8b/biLiKdIN1KjAnlzcuIN3YPK7GR78JbEW6qfonUnKsbPMl0lXH/sDTwIeqlw/Y/2JSDX9L4F7SzdyTSeW1IqYB/5VLM4cOsc66kuaTEtgs4A2kG+kX5RjmAe8g1fgfJv17fpd0RQjwMeC+XBr7NKmcRERcS7oR/YP8PVxBKhdBap22AnAbqTT3fwxS1hvCD4EP5hZIPyr4GWsS9S99mpnZSOUrBDMzA5wQzMwsc0IwMzPACcHMzLKu6ocwduyascYak9odhplZV3nggdlPRsT4eut1VUJYY41JHHHEde0Ow8ysq3zqUxqq93w/LhmZmRnghGBmZpkTgpmZAV12D2Ewyy33Mhtt9CArr7yw3aEUtmDBGO65Z30WLVq+3aGYmb2i6xPCRhs9yIQJffT1TUJqdHDN5osI5s17CniQOXPqjalmZtY6XV8yWnnlhfT1rdEVyQBAEn19a3TVFY2ZjQxdnxCArkkGFd0Wr5mNDD2REMzMrHFOCCW666472HPPHVh//RX58Y//p93hmJkNS9ffVO4k48atztFH/4g///n37Q7FzGzYRlRCGPeObRj15OOvmr9kzbV49qJZDW9//Pi1GD9+LS6++E8Nb8vMrNVGVEIY9eTjLFnj1eM7DZYkzMxGGt9DMDMzwAmhYdOn/5hdd92SXXfdkkcffbjd4ZiZLbMRVTJqhgMP/CwHHvjZdodhZtYwJ4QSPfbYo7zjHVOYN+95Ro0axUkn/S9XXXUbfX2rtDs0M7O6RlRCWLLmWkO2MirD2mu/lhtvfLCUbZmZtdqISghlNC01M+tVvqlsZmaAE4KZmWVtSwiSJki6TNJtkm6VdEi7YjEzs/beQ1gEfCUirpfUB8yWdHFE3NbGmMzMRqy2XSFExCMRcX1+Pw+4HVivXfGYmY10HXEPQdIk4M3AzEGWHSTpOknXzZ//RKtDK+SQQw5g8uS1mDp1i3aHYma2zNqeECSNBc4GvhgRzw9cHhEnRcSUiJgyduyrB6Ybroja08ti333358wzL2h8Q2ZmbdTWhCBpeVIyOCMiftfs/Z12GpxwwtIkEJGmTzutse3usMNUxo1bvfEAzczaqJ2tjARMB26PiO83e38RMH8+nHPO0qRwwglpev78cq4UzMy6WTtbGe0IfAy4WdINed7hEXF+M3YmwcEHp/fnnJNeAHvtleb7ufdmNtK1LSFExFVAS0/DlaRQSQbgZGBmVtH2m8qtVCkTVau+p2BmNpKNmIRQfc9gr73goovS3+p7CsvqU5/6MO961w7cffedvOlN63PGGdPLC9zMrEVGzGinEowd2/+eQeWewtixjZWNfvrTX5cTpJlZG42YhACw337pSqBy8q8kBd9DMDMbQSWjioEnfycDM7OkJxJCdNld4W6L18xGhq5PCAsWjGHevKe65iQbEcyb9xQLFoxpdyhmZv10/T2Ee+5ZH3iQlVfuzIHvBrNgwZgct5lZ5+j6hLBo0fLMmbNhu8MwM+t6XV8yMjOzcjghmHWhZgzjbuaEYNZlZsyASy7pP4z7JZek+WaNGPIegqRzC3z+6YjYv7xwzKyWCHjxRbj22jS9++4pGVx7LWy7bf+Ol2bDVeum8uuAT9RYLuDH5YZjZrVIKQlASgKVxLDttml+LyaDgUnOSa95aiWEIyLiiloflvTNkuMxszoqSaGSDKB3k8GMGemKqHJ8lfLYiivC1Kntjq73DHkPISJ+W+/DRdYxs3JVTorVqu8p9Irq8ljl+CrlsRdf7L3j7QS17iH8ERjyK4+I9zYlIjMbUvVJsVImqkxDuVcK7S7VlFkea/exdItaJaP/yX/3Bl4LnJ6nPww81sygzGxwUiqXVJ8UKyfNFVcs7yTXKaWaMspjnXIs3WDIhFC5fyDp2IiYUrXoj5Kua3pkZjaoqVNfPYx72VcGndKSaajyWNHj7aRj6QZFhq54jaSNIuIeAEkbAq9pblhmI1u9Ekczh3HvlJZMwymPDfV9dcqxdIsiHdO+BFwu6XJJVwCXAV9sblhmI1cndDyrPpFWtPoEOlR5bNtt+5fH6n1fnXAs3aLuFUJEXCBpE2DzPOuOiHixuWGZjUydUuJotFRTlnrlsSLfVyX2au04lm5QNyFIWhn4MjAxIj4paRNJm0XEec0Pz2xk6YQSRytbMhVRqzxW7/uCzjqWTlekZHQK8BKwQ55+CDiqaRGZjXBllTiWLKk9XWv/K64I22zTv1SzzTbltmQqqt5AfrW+r6JlJw8WmBRJCBtHxDHAywARsYA0bIWZNUEZHc+mT4cTTliaBJYsSdPTp5cXZysUuZ9S7/uaOrV/Qq0khUqT0064Z9MpiiSElyStRO6kJmljwPcQzJpgYLnm8MPT3+reuvUsWQILF8IddyxNCieckKYXLqx/pVCpy8+a1b+H8KxZre0hXKSnctHva6iyk3tD91ek2emRwAXABElnADsC+zcxJrMRq4yOZ6NGwcEHw09+kpLAIYek+ZttluaPqvMzsNU9hBttMtrI99UJ92w6SZFWRhdJmg1sTyoVHRIRTzY9MrMRqoyOZ1ddBZtumhJCpXfuppum+UV657aqh3C9dYrE0ej3NZIGC6ynbslI0qXAdhHxp4g4LyKelHRSC2IzG7Ea6XgWkUpD55+fykMR6e/556f5Rcogjd7HKFruKVoSqhdHo9/XSBgssIgiJaMNga9J2iYiKsNdT6n1AbNO1eggZ42UQMq0ZEn/0k/1dATMmQMvvQQrrADjx8MTT6TpOXP6/xofLM5W9hBud5PRTmti225Fbio/C+wGrC3pj5JWbXJMZk3RaGuSIp9vRYuVei2IRo2ClVaCyZNTMoD0d/LkNH/UqNpxtrKHcBlNRhvRin10kyIJQRGxKCI+A5wNXAWs1dywzMrVaGuSskogjSraguiAA9I9g2qbbprmF4mzXlPNsso9jTYZLUMr9tEtipSMTqy8iYhTJd0MfLZ5IZmVr2gJo5klkDJ+bVZaEFWSQKUF0eabL21BVN1MdGAZpHIcjcZZRg/hyjozZ8J22y1dZ+bMpeu06hd6MwcL7CZDXiFIWiW/PUvS6pUXcC9waEuiMytRvRJGs0sgZakkhWrVzUmLlEEa/S7qHWvRGB54AFZbDXbbLU3vtluafuCBNO1OY61Vq2T0q/x3NnBd/ju7atqsq9QqT7SqBFKGSpmoWvU9BShW8mnkuyhyrEVi2GADePZZuPTSNH3ppWl6gw3S8bjTWGsNWTKKiHfnvxu2Lhyz5ijSmqSsEkgzW6xU3zOolIkq0yec8OorhWrVZbF6pZp6JaUi26hcBQwWQ+V9vf2401hr1SoZbVXrVcbOJf1c0uOSbilje2ZDabSMUvTzzW6xMmoUjBnT/57BwQen6TFj6vdCrnwX9Uo19UpKRbZRRJH9NLsEZ0vVuql8bP47htTv4EZAwBtJJaMdhvjccJwKHA+cVsK2zGoqMrZ+rXHzi/SIbfbjLQEOPLB/v4NKUiiSDGBpqebaa1OJZvfdl5Zqij5DoMg2ihxzve+83nIrV62S0a4Akn4HbBURN+fpLUjjGzUsImZImlTGtsyKqFdGKaPcU0bHtHrbGHjyL5oMoJwWQmWUc+p957vtlpKMO421TpFmp5tVkgFARNwi6XVNjKkfSQcBBwGsvvoGrdqtjTBDlXtgabmnyNg806en/gCVX+yVmv+YMemXfRFF9lPG8dYav6fIgHH1tlEkhlr7GTWq8YH+bHiKJISbJZ0MnJ6nPwrc1LyQ+ouIk4CTACZOnOJ2BdY0tco91S1voP+v1UqJpDKGUPUN3uobwAOHmxhMkf2UcSIsozxWRjmn3n5aUYKzpYokhP2Bg4HcBYYZwAlDrm3WxYYqKVVORBH9SyTVTxWTig07Xasc1IqWNUXLY7VaCJVZYqvXKcydxlqn5u8VSaOB6RHxg4jYK79+EBELWxSfWce48sr68yvDTle31a8MOw2Nd/gqQxmtoVrRospar2ZCiIjFwERJKzRj55J+DfwN2EzSg5IKVlnNWqtSDrr0Upg3L82bNy9NV4aUrqwz1LDTRTtataJzWxnj93gMoN5TpGR0D3C1pHOBFyozI+L7je48Ij7c6DbMylRrWGlYej+h8hpYRqk37HS9slPRUkwZw3CXUYpxOae3FGms9g/gvLxuX9XLrKfUGlZaSi2F3vY26OtL0319aXrMmDRdZNjpSnmp+moAls4vUorplGG4rffUvUKoeiiOWc8aOKz0YC2EdtoJLr64f4kE0vyKAw5I68yatXTeppvC29++tKT0l7+k9319qez0l7+kxFL5Bd9oaydoTUsl6z11E4Kk8cBXgdeTei0DEBFva2JcZq9SRoevodQbVlqqP6Q01F5nt93SOpUyVKUlT/WgdBX1WjtBe4fhtt5UpGR0BnAHsCHwTeA+YFatD5iVrRUlkFrDSktpjJ5x4/qP3TNuXP/xf2qVe0aNgrlz05APfbno2teXpufOHV4zzU4Yhtt6T5GEsEZETAdejogrIuIAwFcH1jKteBIZ1B5WujJ2zzPP9B+q+Zln0vwiQz5HwIQJ6eRf3VJp7tw0v+hxdMow3NZ7irQyejn/fUTSvwIPA6s3LySz/lrRWavIsNKDxVAZ+nk4rXdqtVSqp0grJPCD423ZFLlCOErSqsBXSE9KOxn4UlOjMhug2SWQyrDS1b2KDz44TVeGlW40hiItlYpsoxOG4bbeVKSV0Xn57XPArs0Nx2xwrRgGeZNN4J//7F/u2WST1GS0rBiKtFSqp1OG4bbeM2RCkHQcMGTFMSK+0JSIzAYoc9ycWvt48UW47rp0NVDZx3XXpX0uWdL4UMyV46jVUqms8X+KrmNWrdYVQuW5yTsCk4Hf5Ol9gNuaGZRZtaFKIFBeCaTI4HWNxtCK4zBrhKJOswNJ1wBvjYhFeXp54MqI2L4F8fUzceKUOOKI6+qvaD2pmf0QIDVhXbhw6S/2iHTyHjOm/4PhG42h2cdhNtCnPqXZETGl3npFbiqvBqxSNT02zzNrqWaWQKp7Ec+bl6YrvYgrg9eVFYNLOdapijQ7/Q7wd0mXAQKmUtIjNM06TeXXeuU18ALav+6tlxVpZXSKpD8D2+VZX4uIR5sblllrVZqE7rbb0nGI+vrSPYRKk9BWPNrSrJ2KPpp7NPAE8AywqST/5289Z6imnzvt1Lre0mbtVGRwu+8CHwJuBSrDcAXpUZpmhXR6qaVIk9CivaU7/VjNhlLkHsL7gc0i4sVmB2O9qRtKLUWbhO6++9JkUJmuPtl3w7GaDaVIyegeYPlmB2K9qZtKLfUeCVlvwLhuOlazwRS5QlgA3CDpUuCVqwT3VLYiWjEwXZmGahJatLd0Nx2r2UBFEsK5+WW2TConylqllk5XtKTUC8dqI1eRZqe/aEUg1rtaMTBdKxQZMK5XjtVGpiKtjDYBppHGM6p+hOZGTYzLukitVjWVE+TMmUufHVCZhu47UdbqZdyKQfjMqk2bBjzxWGnbK1IyOgX4BvAD0vDXH6d4/wXrcfVa1VQePbnaav0fPTlnztJHT/YKD15nZZoxA64+66HaKy1ezL1b71N3W7q27ipAsYSwUkRcKkkRcT9wpKTZwNeL7cJ6VXWrGuj/i3jbbZe2qtlggzTv0kvTOpdeCs8+u3SdXjpR+jkEVsS0zz8EixfXXe+wcSdy0B4P1F5p6oH1d3htsYxQJCG8KGkUcJekzwEPkQa4sxGuaKuakdbyxoPXjVxFT/RrjX6KmT+ZVWCLk/KrNYokhEOAlYEvAN8mlY32a2ZQ1j2KtKpxyxvrdtMOfz5d1hZw708uaHI0zVMkIUyKiFnAfNL9AyTtA8xsZmDWHYq0qnHLG2unGXUG2bnrLnh8du2yzFqjn2Lmvsf2fHfzIgnhMOCsAvNshCnSqgbc8saaY/r0+us8PvsB1hr9VN317p2wNxx2WJ21ejsZQO1nKu8JvAtYT9KPqhatAixqdmDW+Yq2qnHLGxuOadMKrDQ3/aLfcczsmqsdOG4WBx09qcAG6yWDkaHWFcLDpOcqvxeo/tbnAV9qZlDWPYq0qnHLG6uYPh0e/8fzQ68wbx4sXsxh406svaFxFDzRF1nHKoZMCBFxI3CjpF9FxMsAklYDJkTEM60K0DpfkVY1bnnT22bMgKt/X6CD1MIXuXfCzkMv7yOXbiaVFJkNR5F7CBdLem9edzbwuKS/RoSvEqw0foZA55r2+TqdowAWL2bHMbM5/fXfqb/ugS7PdKoiCWHViHhe0ieA0yLiG5JuanZgNnL4GQLtMe3Lj8HC+o85WWv0U8zc59j6G5w6FSjQSco6VpGEsJykdYB/A45ocjw2whTp7ewrheGbNo1XbrzWUrzNvDPzSFAkIXwLuBC4KiJmSdoIuKu5YdlI4WcI9FdGm/mKe7feBw70L3Yrrsjw12dR1ecgIu4BPtDMoGxkGQk9meud6AGuPrNYm/nDxp1VsIWNk4ENT61+CAdFxEm1PlxknTqf3wP4ITAaODkiCtyRsl7T7T2Z63WQqvyir9dmfscxcPr3nyiwx0nFAjMbplpXCP8p6ckay0Ua52iZEoKk0cCPgbcDDwKzJJ0bEbcty/asO3XyMwSmT4fHH6+z0hPpxmytk/0mhU/0Zu1VKyFcAbynzucvbmDf2wJ35xIUks4E3gc4IYwg7XqGwLTDa3SOqnj22fodpMbAQe9/wM2hrCfU6pj28Sbvez1gbtX0g8B2A1eSdBBwEMDqq2/Q5JCsHcrsyTzty8U6R+04Zjanj/9y7fU2XqvgTdlJRUIz63hFWhm1Vb5HcRLAxIlTos3hWJMU6ck87fDn09AGQ8nj0Bd5glQ60buDlFm1diaEh4AJVdPr53k2whRtMw9w776H117BnaPMllk7E8IsYBNJG5ISwb7AR9oYj5VsxozUlLKIeyfsXGD4YXAHKbPmqZsQJK1I6ncwqXr9iPhWIzuOiEX5kZwXkpqd/jwibm1km1aOom3mizhs3IkeftisSxS5QvgD8BxpYLv6A58MQ0ScD5xf5jatthkzUm/XoTx+Q3ombL0OUjuOuc9t5s16TJGEsH5E7NH0SKwhlRN9zXbz89JzYWu2mV8eTj/ObebNRqIiCeGvkt4QETc3PRobVNEnSNVtMw8ctK/bzJvZ4GoNXXEzEHmdj0u6h1QyEhAR8cbWhNi7inaOWmv0UxzYV/sR1gdN+HXBm7KTCsVmZiNPrSuEd7csih407fDnYeE/h14hj0Nf8+lRMIwnSPmmrJk1plZP5fsBJP0yIj5WvUzSL4GPDfrBHjd9er7xWs/ixfU7SLlzlJl1kCL3EF5fPZEHpdu6OeG0z7DbzO+4Y+2V3EHKzLpMrXsIhwGHAytJqhS7BbzEMo5w2i7TPlPsRL/jmNkFm1L6V72Z9Z5aJaNpwDRJ0yKiI86A8+e/utPU1b+v/1zYtUY/xczjZjUxMjOz7lekZHSWpK0GzHsOuD8iFjUhpiEtfGYBd531937z1gJm/sQnezOzRhVJCD8BtgJuIpWM3gDcAqwq6eCIuKiJ8fXzugkvMPMIn/zNzJphVIF1HgbeHBFTImJrYEvgHtKTzo5pZnBmZtY6RRLCptWDzuVHXG5eedKZmZn1hiIlo1slnQCcmac/BNyWR0F9uWmRmZlZSxW5QtgfuBv4Yn7dk+e9DOzarMDMzKy16l4hRMQ/gWPza6D5pUdkZmZtUeQBOTsCRwIT6f+AnI2aF5aZmbVakXsI04EvkR6Qs7i54ZiZWbsUSQjPRcSfmx6JmZm1VZGEcJmk7wG/o+oRmhFxfdOiMjOzliuSELbLf6dUzQvgbeWHY2Zm7VKklZGblpqZjQBFWhmtDRwNrBsRe0qaDOwQEdObHp2139FHw7x5r57f1weHH949+zCzuop0TDsVuBBYN0/PIXVQs5Fg3jwYO/bVr8FO4J28DzOrq0hCWDMifgssAchDXrv5qZlZjymSEF6QtAbpRjKStic9D8HMzHpIkVZGXwbOBTaWdDUwHvhgU6MyM7OWK9LK6HpJOwObkR6Qc2dEeJRTM7MeM2RCkLT3EIs2lURE/K5JMVkn6esbugVQWebPh2efffX85YpcwA7D5z8PiwZ56utyy8Fxx5WzD7eYsi5W6/+499RYFqSey9brWnESq7QqGmh+yYPpLlo0eJIZLEksq0qLqcHmm3W4IRNCRHy8lYGYmVl7lXxNbl2lSHmjjDLLpz8NEa+eL8GJJ8Jzz6WXmbWVE8JIVqS8UUaZJSKd/AebX/nb7FKOmdVVpB+CmZmNAMvSygjArYy6QStavNQrB1UMtk7FkiXw0ku191OvdFXkWIvsp1GtajFl1gRuZdTLWtHipV45qCz1SldFjnXUqOaXplrVYsqsCdzKyMzMgII3lSX9K/B6YExlXkR8a1l3Kmkf4EjgdcC2EXHdsm6rZ5VR7nn66fQaSpFOZ0XKLGVfDQymFeUedyqzEa7I8xBOBFYGdgVOJo1jdG2D+70F2Bv4aYPb6V1llXtqlXOKnOTKKrOUUVZqdmnKncpshCvSyugtEbEf8ExEfBPYAdi0kZ1GxO0RcWcj2zAzs3IVKRn9M/9dIGld4ClgneaF1J+kg4CDADZYffVW7bZ3tKKcU0QZcdTaRpHy13LLDd1SqSytGPvJrEmK/J9wnqRxwPeA60ktjE5SfNc5AAAMAklEQVSu9yFJlwCvHWTRERHxh6IBRsRJwEkAUyZO7JCzWwvUq/9D/Zp3vXJPGTXz1VdvvFVNkbJUvf0UiXeddWqfrMvoMe17DdbFiiSEYyLiReBsSeeRbiwvrPehiNi90eBGvHo180Zr3q2smbeiaWo99Y7XPaZthCuSEP4GbAWQE8OLkq6vzLMmavSEWUaJpN42ipZIah1LkThbUYqRYPEgT4cdLJmZ9aBaPZVfC6wHrCTpzaSH4wCsQmp1tMwk7QUcR3r62p8k3RAR72xkmz2njNY9ZYzxX28bRUoko0c3to+i+2nUqqs2v1OZm7ZaB6v1U/GdwP7A+sD3q+Y/DzT0X25EnAOc08g2rIu4FLOUm7ZaB6vVU/kXwC8kfSAizm5hTGZm1gZFislXS5oOrBsRe0qaDOwQEdObHFtvq1c6aEVdvVVNJDulNl/veN1k1Ea4IgnhlPw6Ik/PAX4DOCE0ol7poBV19VbVrFtRmy+i3vG6hm8jXJGeymtGxG+BJQARsQgY5OeemZl1syJXCC9IWoPUIQ1J2wN+3mE36JQWLS7FLOXvwjpYkYTwZeBcYGNJV5Oain6wqVFZOTqlRYtLMUv5u7AOVjchRMT1knYGNiP1RbgzIl5uemRmZtZSRYa/HgN8BngrqWx0paQTI6Lu8BVWg0sHZtZhipSMTgPmkXoWA3wE+CWwT7OCGhFcOjCzDlMkIWwREZOrpi+TdFuzAjIzs/YokhCul7R9RFwDIGk7wI+87AYuS5nZMBRJCFsDf5X0QJ7eALhT0s1ARMQbmxadNcZlKTMbhiIJYY+mR2FmZm1XpNnp/a0IxMzM2qvI0BVmZjYCOCGYmRnghGBmZpkTgpmZAU4IZmaWOSGYmRnghGBmZpkTgpmZAU4IZmaWOSGYmRnghGBmZpkTgpmZAU4IZmaWOSGYmRnghGBmZpkTgpmZAU4IZmaWOSGYmRnghGBmZpkTgpmZAU4IZmaWOSGYmRnQpoQg6XuS7pB0k6RzJI1rRxxmZrZUu64QLga2iIg3AnOAw9oUh5mZZW1JCBFxUUQsypPXAOu3Iw4zM1uqE+4hHAD8eaiFkg6SdJ2k656YP7+FYZmZjSzLNWvDki4BXjvIoiMi4g95nSOARcAZQ20nIk4CTgKYMnFiNCFUMzOjiQkhInavtVzS/sC7gd0iwid6M7M2a1pCqEXSHsBXgZ0jYkE7YjAzs/7adQ/heKAPuFjSDZJObFMcZmaWteUKISL+pR37NTOzoXVCKyMzM+sATghmZgY4IZiZWeaEYGZmgBOCmZllTghmZgY4IZiZWaZuGjVC0hPA/e2OY5jWBJ5sdxBN0IvH1YvHBD6ubtOM45oYEePrrdRVCaEbSbouIqa0O46y9eJx9eIxgY+r27TzuFwyMjMzwAnBzMwyJ4TmO6ndATRJLx5XLx4T+Li6TduOy/cQzMwM8BWCmZllTghmZgY4IbSEpO9JukPSTZLOkTSu3TE1StI+km6VtERS1zf9k7SHpDsl3S3pP9sdTxkk/VzS45JuaXcsZZI0QdJlkm7L/w0e0u6YyiBpjKRrJd2Yj+ubrY7BCaE1Lga2iIg3AnOAw9ocTxluAfYGZrQ7kEZJGg38GNgTmAx8WNLk9kZVilOBPdodRBMsAr4SEZOB7YHP9si/14vA2yLiTcCWwB6Stm9lAE4ILRARF0XEojx5DbB+O+MpQ0TcHhF3tjuOkmwL3B0R90TES8CZwPvaHFPDImIG8HS74yhbRDwSEdfn9/OA24H12htV4yKZnyeXz6+WtvpxQmi9A4A/tzsI62c9YG7V9IP0wAlmJJA0CXgzMLO9kZRD0mhJNwCPAxdHREuPqy3PVO5Fki4BXjvIoiMi4g95nSNIl7tntDK2ZVXkmMzaRdJY4GzgixHxfLvjKUNELAa2zPcZz5G0RUS07B6QE0JJImL3Wssl7Q+8G9gtuqTzR71j6iEPAROqptfP86xDSVqelAzOiIjftTueskXEs5IuI90DallCcMmoBSTtAXwVeG9ELGh3PPYqs4BNJG0oaQVgX+DcNsdkQ5AkYDpwe0R8v93xlEXS+EoLREkrAW8H7mhlDE4IrXE80AdcLOkGSSe2O6BGSdpL0oPADsCfJF3Y7piWVb7h/zngQtINyt9GxK3tjapxkn4N/A3YTNKDkg5sd0wl2RH4GPC2/P/TDZLe1e6gSrAOcJmkm0g/Ui6OiPNaGYCHrjAzM8BXCGZmljkhmJkZ4IRgZmaZE4KZmQFOCGZmljkhWMeRtIukVzW3G2p+Cft7f/XgaJIurzeCa47lOUnn11nv8LLizNubX3+tmp/fX9Lx+f2nJe1XQkz3SVpT0kq5CehLktZsdLvWek4IZvB+0iinw3VlRNRr/15qQhgOJUP+Px4RJ0bEaWXtLyL+GRFbAg+XtU1rLScEGzZJr5H0pzxu+y2SPpTnby3pCkmzJV0oaZ08/3JJP8y/Hm+RtG2ev62kv0n6u6S/StpsmDH8PI8f/3dJ78vz95f0O0kXSLpL0jFVnzlQ0pz8mZ9JOl7SW4D3At/L8W2cV98nrzdH0k4F4llH0oyqY9xJ0neAyq/mM/J6v8/fz62SDqr6/HxJ/52/02skrZ3nb5i/o5slHVW1/lhJl0q6Pi+rHP8kpec6nEYa8mCCpI9XjpvUqauyjSMlHSpp3aoOXjdIWixpYu45e7akWfm1Y/7cGpIuysdwMqCi/27W4SLCL7+G9QI+APysanpV0lC9fwXG53kfAn6e319eWR+YCtyS368CLJff7w6cnd/vApw3yH5fmQ8cDfx7fj+O9JyJ1wD7A/fkmMYA95PGKVoXuA9YPcd6JXB8/vypwAer9nM5cGx+/y7gklqx5OmvkAb9AxgN9OX38wd8bvX8dyXSCXuNPB3Ae/L7Y4D/yu/PBfbL7z9b2R5pHLJV8vs1gbtJJ+ZJwBJg+7xsHeABYDywAnB11XEfCRw6IL7PknpqA/wKeGt+vwFpqAiAHwFfz+//Nce+ZtU27que9qt7Xh7czpbFzcCxkr5LOileKWkLYAvS8ByQToqPVH3m15DG6Je0itKYLX3ALyRtQjqpLD+MGN4BvFfSoXl6DOmkBXBpRDwHIOk2YCLppHlFRDyd558FbFpj+5UB02aTTrL1zAJ+rjTo2u8j4oYh1vuCpL3y+wnAJsBTwEtA5f7IbNI4NpB+0X8gv/8l8N38XsDRkqaSEsB6wNp52f0RcU1+vx1weUQ8ASDpNwxx3PkK4JPAW/Os3YHJ+d8TYBWlEUankh6ORET8SdIzQxyrdRknBBu2iJgjaSvSr+ejJF0KnAPcGhE7DPWxQaa/DVwWEXspjWt/+TDCEPCBGPCQHknbkZ48VbGYZfvvvLKNQp/PiW4q6RfzqZK+HwPq85J2IZ1kd4iIBZIuJyUygJcj/7weZJ+DjS/zUdKv/q0j4mVJ91Vt64V68Q6Uy3vTSQMwVm5cjyJdaSwcsO5wN29dwvcQbNgkrQssiIjTge8BWwF3AuMl7ZDXWV7S66s+VrnP8FbgufwLflWWDjO9/zDDuBD4vPLZSdKb66w/C9hZ0mqSlmPpr26AeaSrlWUmaSLwWET8DDiZ9J0AvJyvGiAd7zM5GWxOevxjPVeTRl+FlAQqVgUez8lgV9JV0GBmko57jRzHPoPEvjxwFvC1iJhTtegi4PNV622Z384APpLn7QmsVuA4rAs4IdiyeANwrdKTnb4BHBXp0ZMfBL4r6UbgBuAtVZ9ZKOnvwIlAZdTNY4Bpef5wf8V/m1RiuknSrXl6SBHxEOm+w7Wkk+x9wHN58ZnAf+Sb0xsPvoW6dgFuzMfyIeCHef5JOcYzgAuA5STdDnyH9DjVeg4hPTP4Zvo/xe0MYEqevx9DDJMcEY+Q7hX8jXTctw+y2luAKcA3q24srwt8Ie/jplx6+3Re/5vA1Py97026R2E9wKOdWtPl0sihEXFdm+MYGxHz8xXCOaSb3ucs47Z2IR3Tu8uMsRfk8tWUiHiy3bHY8PgKwUaSI/NVzS3AvcDvG9jWS8AWqtMxbSRR7phGunJb0u54bPh8hWBmZoCvEMzMLHNCMDMzwAnBzMwyJwQzMwOcEMzMLPv/HaKq8rfiJyoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f51506bed10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEKCAYAAAAfGVI8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3Xl8leWZ//HPdU5WQiAsIYGAgoAoooBG61ZHsQptXRi11bbTn1qntp2O2tpadZZ2lk7rlG5ara2ttnbGUpfiUlulDKDWvUF2kU1BCGGVsIas1++P8wQDZDmJOXnOyfN9v17ndZ7znO2LQq7cy3Pf5u6IiEh0xcIOICIi4VIhEBGJOBUCEZGIUyEQEYk4FQIRkYhTIRARiTgVAhGRiFMhEBGJOBUCEZGIywo7QDIGDx7sI0eODDuGiEhGWbBgwXZ3L+7odRlRCEaOHElFRUXYMUREMoqZrU/mdeoaEhGJOBUCEZGIUyEQEYk4FQIRkYhTIRARibiUzhoys3XAHqARaHD3cjMbCDwMjATWAZ90953d/d1PLKxkxuyVbKquYVhRPrdMHcf0yWXd/TUiIhmvJ1oE57n7JHcvDx7fBsx197HA3OBxt3piYSW3z1pKZXUNDlRW13D7rKU8sbCyu79KRCTjhdE1dCnwYHD8IDC9u79gxuyV1NQ3HnKupr6RGbNXdvdXiYhkvFQXAgf+bGYLzOz64FyJu1cFx5uBktbeaGbXm1mFmVVs27atU1+6qbqmU+dFRKIs1YXgbHc/Gfgo8GUzO6flk+7uJIrFEdz9Pncvd/fy4uIOr5A+xLCi/E6dFxGJspQWAnevDO63Ao8DpwFbzGwoQHC/tbu/95ap48jPjh9yLj87zi1Tx3X3V4mIZLyUFQIzKzCzwuZj4EJgGfAUcHXwsquBJ7v7u6dPLuO7l51IWdACiJvxnb+doFlDIiKtSOX00RLgcTNr/p7fuvuzZvZX4BEzuw5YD3wyFV8+fXIZ0yeX8UjFBr7x2BJGD+mbiq8REcl4KSsE7v42MLGV8zuA81P1vYe74PgS4jHjmWWbOWl4UU99rYhIxuj1VxYPKMjhjGMG8eyyzSTGpkVEpKVeXwgApk4o5Z3t+1i1ZW/YUURE0k40CsH4Eszg2WWbw44iIpJ2IlEIhvTL45SjBvDMsqqOXywiEjGRKAQA0yaU8tbmPazbvi/sKCIiaSUyhWDqCaUAPLtc3UMiIi1FphCMGNiHE8v6a5xAROQwkSkEkOgeWrShmqpdWnxORKRZ5AoBaPaQiEhLkSoEo4v7cmxJXxUCEZEWIlUIAKadUMpf173H9r21YUcREUkL0SsEE4bS5DDnzS1hRxERSQuRKwTHDy3kqIF9eEbdQyIiQAQLgZnx0QmlvLxmO7tq6sOOIyISusgVAkjMHmpocuauUPeQiEgkC8HE4UWU9svT7CERESJaCGIxY9qEUp5ftY19tQ1hxxERCVUkCwEk1h6qbWjiuZXbwo4iIhKqyBaC00YNZFBBjhahE5HIi2whiMeMC8aXMG/FFg7UN4YdR0QkNJEtBJCYPbSvrpGX1mwPO4qISGgiXQjOHD2YwrwsXVwmIpEW6UKQkxXjI8eX8H8rtlDf2BR2HBGRUES6EECie6h6fz2vvf1e2FFEREIR+UJwzthi8rPjPLtcG9uLSDRFvhDk58Q577hiZi/fQlOThx1HRKTHRb4QQOLism17alnw7s6wo4iI9DgVAmDKcUPIice09pCIRJIKAVCYl82Hxw7m2WWbcVf3kIhEiwpBYOqEUiqra1hWuTvsKCIiPUqFIHDB8SXEY8YzyzR7SESiRYUgMKAgh9OPGajuIRGJnJQXAjOLm9lCM3s6eDzKzF4zszVm9rCZ5aQ6Q7KmTRjK29v3sXrr3rCjiIj0mJ5oEdwErGjx+L+BH7n7GGAncF0PZEjK1PElmMEzSzV7SESiI6WFwMyGAx8Hfhk8NmAK8FjwkgeB6anM0BlD+uVxylEDtEeBiERKqlsEPwa+ATSv6DYIqHb35v0hNwJlKc7QKdMmlLKiajfrd+wLO4qISI9IWSEws4uAre6+oIvvv97MKsysYtu2nttOcuoJpQC6uExEIiOVLYKzgEvMbB3wOxJdQncCRWaWFbxmOFDZ2pvd/T53L3f38uLi4hTGPNSIgX04say/9igQkchIWSFw99vdfbi7jwSuAua5+2eA+cAVwcuuBp5MVYaumjahlEUbqqnaVRN2FBGRlAvjOoJbgZvNbA2JMYP7Q8jQrmkTEt1Ds9UqEJEI6JFC4O7PuftFwfHb7n6au49x90+4e21PZOiM0cV9GTukr2YPiUgk6MriNnx0Qimvv/MeO/amXZ0SEelWKgRtmDqhlCaHP7+5JewoIiIppULQhvFD+3HUwD6aRioivZ4KQRvMjI9OKOXltdvZVVMfdhwRkZRRIWjH1Aml1Dc6895S95CI9F4qBO2YNLyI0n55WoRORHq1DgtBsIz0/J4Ik25iMWPqCSU8v2ob++saOn6DiEgG6rAQuHsj0GRm/XsgT9qZNmEotQ1NPLey59Y7EhHpSVkdvwSAvcBSM5sDHFyW091vTEmqNHLqyAEMLMjh2WWb+diJQ8OOIyLS7ZItBLOCW+RkxWNcOL6Ep5dUUdvQSG5WPOxIIiLdKqlC4O4PBltKHhucWunukZlTOW1CKb/76wZeXL2d848vCTuOiEi3SmrWkJmdC6wG7gF+Cqwys3NSmCutnDl6MIV5Wbq4TER6pWS7hn4AXOjuKwHM7FhgJnBKqoKlk5ysGB85voQ5K7ZQ39hEdlyzbkWk90j2J1p2cxEAcPdVQHZqIqWnqSeUUr2/ntffeS/sKCIi3SrZQlBhZr80s3OD2y+AilQGSzd/c2wx+dlxnllWFXYUEZFulWwh+BLwJnBjcHszOBcZ+Tlxzh1XzOzlW2hq8rDjiIh0mw7HCMwsDjwQbDP5w9RHSl/TJpTyzLLNvPHuTspHDgw7johIt0j2yuKjg+mjkTbluCHkxGOaPSQivUqys4beBl4ys6c49MriSLUQCvOyGTOkgF+9tI77X3yHYUX53DJ1HNMnl4UdTUSky5ItBGuDWwwoTF2c9PbEwkpWb91LoyfGCCqra7h91lIAFQMRyVjJjhEUuvvXeyBPWpsxeyX1jYcOFNfUNzJj9koVAhHJWMmOEZzVA1nS3qbqmk6dFxHJBMl2DS0Kxgce5dAxgkgtRDesKJ/KVn7oDyvKDyGNiEj3SPY6gjxgBzAFuDi4XZSqUOnqlqnjyM8+dPXRvOwYt0wdF1IiEZEPLtnVR69NdZBM0DwOMGP2SjZV1+DAuccWa3xARDJauy0CM3ukxfF/H/bcn1MVKp1Nn1zGS7dN4Z07Ps4F40t4ee0O9hyIzIrcItILddQ1NLbF8QWHPVfczVkyzo1TxrL7QAO/eWV92FFERLqso0LQ3qI6kV9w58Th/Tl3XDH3v/iONrcXkYzVUSHoY2aTzewUID84Prn5cQ/kS3s3TBnLe/vqeOjVd8OOIiLSJR0NFlfx/kJzmzl00TktuAOccvQAzhoziJ+/8DafPeNo8rK1p7GIZJZ2C4G7n9dTQTLZDVPGctV9rzLz9Xe59qxRYccREemUTu+5aGb3pSJIJjv9mEGcNnIgP3/+bWobGsOOIyLSKV3ZfLe821P0AjecP4bNuw/waMXGsKOIiHRKVwrB1mReZGZ5Zva6mS02s+Vm9u/B+VFm9pqZrTGzh3vLPgdnjxnMpBFF3PvcWuobm8KOIyKStE4XAnefluRLa4Ep7j4RmARMM7PTgf8GfuTuY4CdwHWdzZCOzIybzh9LZXUNj79RGXYcEZGktTtYbGZ/oJ3rBdz9knaec2Bv8DA7uDmJ9Yo+HZx/EPg34N6kE6exc8cVc2JZf+55bg2XnVxGVrwrDS4RkZ7V0U+q7wM/AN4BaoBfBLe9JDaqaZeZxc1sEYnupDnBe6rdvfnqq41Ar1mox8z4xyljWL9jP39YsinsOCIiSelo+ujzAGb2A3dvOUj8BzOr6OjDg70MJplZEfA4cFyywczseuB6gKOOOirZt4XuguNLOK60kLvnreGSiWXEYxZ2JBGRdiXbd1FgZsc0PzCzUUBBsl/i7tXAfOAMoMjMmgvQcKDVDnV3v8/dy929vLg4c5Y1isUSrYK12/bxzLKqsOOIiHQo2ULwVeA5M3vOzJ4n8UP9K+29wcyKg5YAZpZPYtG6FcF7rwhedjXwZFeCp7OPThjK6OIC7p63hqamyC/JJCJpLqlC4O7PkliJ9CbgRmCcu8/u4G1DgflmtgT4KzDH3Z8GbgVuNrM1wCDg/q6GT1fxoFXw1uY9zFmxJew4IiLtSmpjGjPrA9wMHO3unzezsWY2LvjB3ip3XwJMbuX828BpXQ2cKS4+aRg//r/V3DV3NReOL8FMYwUikp6S7Rr6FVBHoo8fEv36305Jol4iKx7jy+eOYfmm3cxfmdQ1eCIioUi2EIx29+8B9QDuvh/Qr7gd+NuTyygryueuuWtIXFYhIpJ+ki0EdcGArwOY2WgSVw5LO7LjMf7hvNEs2lDNi2u2hx1HRKRVyRaCbwHPAiPM7CFgLvCNlKXqRa44ZThD++fxk7lrwo4iItKqDguBJUY53wIuA64BZgLl7v5cSpP1ErlZcb5wzjG8vu49Xn17R9hxRESO0GEhCNYM+pO773D3P7r70+6ufo5OuOq0oxjcN5efzFsddhQRkSMk2zX0hpmdmtIkvVhedqJV8NKaHSxYvzPsOCIih0i2EHwIeMXM1prZEjNbGlwoJkn6zOlHMbAgR60CEUk7SV1QBkxNaYoI6JOTxXVnj2LG7JUs2VjNScOLwo4kIgIkv8TEendfT2Ipam9xk074f2ccTf/8bH4yTzOIRCR9JFUIzOwSM1tNYl+C54F1wDMpzNUrFeZl87mzRjHnzS28uWl32HFERIDkxwj+EzgdWOXuo4DzgVdTlqoXu+askRTmZnH3fI0ViEh6SLYQ1Lv7DiBmZjF3nw+Ud/QmOVL//GyuPnMkzyzbzOote8KOIyKSdCGoNrO+wAvAQ2Z2J7AvdbF6t8+dPYr87Dh3z9dYgYiEL9lCcCmJgeKvklhqYi1wcapC9XYDC3L47OlH84fFm3hnu+qpiIQr2VlD+9y90d0b3P1Bd78r6CqSLvr7Dx9DTlaMe9QqEJGQJTtraI+Z7Q5uB8ys0cw07eUDKC7M5VOnHcXjCyvZ8N7+sOOISIQl2yIodPd+7t4PyAcuB36a0mQR8IVzRhM346fPrQ07iohEWLJjBAd5whPoauMPrLR/Hp88dTiPLdjApuqasOOISEQl2zV0WYvbFWZ2B3Agxdki4UvnjgHg58+rVSAi4Uh2raGWM4QaSFxZfGm3p4mgsqJ8Lj95OP/76npmL9/Clt0HGFaUzy1TxzF9clnY8UQkApIqBO5+baqDRNnYIX1pdNi8O9HIqqyu4fZZSwFUDEQk5ZIqBGZ2V3vPu/uN3RMnmh54ad0R52rqG5kxe6UKgYikXLKDxXnAycDq4DYJyAEWBDf5ANoaKNYAsoj0hGTHCE4Cznb3BgAz+xnwF3f/YsqSRciwonwqW/mhP6woP4Q0IhI1ybYIBgD9WjzuG5yTbnDL1HHkZ8cPOZebFeOWqeNCSiQiUZJsi+AOYKGZzQcMOAf4t1SFiprmcYAZs1eyqbqGWMzIz45z3rghIScTkSgw9+Q2GjOzUhJ7FzvwurtvTmWwlsrLy72ioqKnvi50izZUc8W9L3P+8UP42d+dgpmFHUlEMpCZLXD3DrcMaLdryMyONrP+AMEP/t0kNqX5tJnldEtSOcKkEUXcOu04Zi/fwv+8uj7sOCLSy3U0RvAIUABgZpOAR4F3gYloraGUuu7sUUw5bgjffnoFyyp3hR1HRHqxjgpBvrtvCo7/DnjA3X8AXAucltJkEReLGd//xEQGFuRww8yF7K1tCDuSiPRSHRWClp3TU4C5AO7elLJEctDAghzuvGoS63fs418eX0qy4zkiIp3RUSGYZ2aPBFtTDgDmAZjZUKAu1eEEPnTMIL7ykWN5YtEmHl2wMew4ItILdVQIvgLMIrHI3NnuXh+cLwX+ub03mtkIM5tvZm+a2XIzuyk4P9DM5pjZ6uBe1yN04MvnjeHM0YP45pPLtOG9iHS7dgtBsPfA79z9R+5eCWBmF7n7Qnef3cFnNwBfc/fxwOnAl81sPHAbMNfdx5Loarrtg/8xerd4zPjxlZMoyMniH3+7kJq6xrAjiUgv0umNaYD/SOZF7l7l7m8Ex3uAFUAZieWrHwxe9iAwvQsZImdIvzx+dOUkVm7Zw388vTzsOCLSi3SlEHT66iYzGwlMBl4DSty9KnhqM1DSxnuuN7MKM6vYtm1bF2L2PuccW8yXzh3NzNc38NTiTR2/QUQkCV0pBF/ozIvNrC/we+Ar7n7IhveemAbT6lQYd7/P3cvdvby4uLgLMXunmy84llOOHsA/zVrKuu37wo4jIr1AsltVxs3sEjO7ETjbzG42s5uTeF82iSLwkLvPCk5vCWYdNc8+2trF7JGUHY9x16cmE48ZN8xcSG2DxgtE5INJtkXwB+AaYBBQ2OLWJksskHM/sMLdf9jiqaeAq4Pjq4EnO5FXSGxvOeOKk1hauYs7nnkr7DgikuGSXX10uLuf1MnPPgv4LLDUzBYF5/6JxEqmj5jZdcB64JOd/FwBLjyhlGvOHMmvXlrHmaMHc8H4VodaREQ6lGwheMbMLnT3Pyf7we7+Im0PLJ+f7OdI227/2HFUrH+Prz+6mD/d9GHKtJGNiHRBsl1DrwKPm1mNme02sz1mtrvDd0lK5WbFuftTJ9PY5Nw4cyH1jVr5Q0Q6L9lC8EPgDKCPu/dz90J379fRmyT1Rg4u4DuXnciC9Tv50ZxVYccRkQyUbCHYACxzrXqWli6ZOIyrTh3BT59bywurdM2FiHROsmMEbwPPmdkzQG3zycNmA0mIvnXxCbzx7k5ufmQRf7rxwwzplxd2JBHJEMm2CN4hsS5QDklOH5WelZ8T555Pn8ze2ga+8vAiGpvUeBOR5CTVInD3f091EPngxpYU8h+XTOAbv1/CT+ev4Ybzx4YdSUQyQFKFwMzm08pSEO4+pdsTyQfyifLhvLx2Oz/6v1WcNmogHzpmUNiRRCTNJTtG8PUWx3nA5SSWmZY0Y2Z8+29PZPHGXdz0u0X86aYPM7AgJ+xYIpLGrKsTgczsdXfvkX2Ly8vLvaKioie+qtdYVrmLS+9+kax4jLqGJoYV5XPL1HFMn1wWdjQR6SFmtsDdyzt6XbJdQwNbPIwB5UD/LmaTHrBm615iMaO2IXGRWWV1DbfPWgqgYiAih0i2a2gB748RNJDYuvK6VASS7jFj9krqGw9t7dXUNzJj9koVAhE5RLuFwMxOBTa4+6jg8dUkxgfWAW+mPJ102abqmk6dF5Ho6ug6gp8DdQBmdg7wXRLbS+4C7kttNPkghrWxAF1+TpwGrUkkIi10VAji7v5ecHwlcJ+7/97d/xUYk9po8kHcMnUc+dnxQ85lxYz9dY1c/z8L2F+nSV8iktBhITCz5u6j84F5LZ5LdnxBQjB9chnfvexEyoryMRKb2Xz/ExP59vQJPLdyK5/6xWvs2Fvb4eeISO/X0Q/zmcDzZrYdqAH+AmBmY0h0D0kamz65rNWB4SGFudwwcyGX3/syD37uNI4eVBBCOhFJF+22CNz9v4CvAb8Gzm6x+mgMuCG10SRVLjyhlN9+/kNU19Rz+b0vs2RjddiRRCREHS465+6vuvvj7r6vxblV7v5GaqNJKp1y9EAe++KZ5GbFueq+V3lu5dawI4lISJJdfVR6oTFD+vL4P5zJyEEF/P2DFTy2YGPYkUQkBCoEETekXx4Pf+F0Tj9mEF9/dDH3zF+D9h8SiRYVAqEwL5sHrjmV6ZOGMWP2Sv71yWXaz0AkQjQFVADIyYrxw09OoqR/Hj9//m227q7lrk9NJu+waxFEpPdRi0AOisWM2z96PN+6eDxzVmzhM798jZ376sKOJSIppkIgR7j2rFHc/amTWbpxF5f/7GU2vLc/7EgikkIqBNKqj580lN9cdxrb99Ry2b0vs3yTrh8U6a1UCKRNpx8ziMe+dCZZMePKn7/KS2u2hx1JRFJAhUDadWxJIbP+4UzKivK55lev88TCyrAjiUg306wh6dDQ/vk88sUzuP43FXzl4UXMW7GFBe/uZFP1AW2BKdILqEUgSemfn82DnzuNScP789SSKiqrD+C8vwWmWgoimUuFQJKWlx1naytLVzdvgSkimUmFQDqlqvpAq+e1BaZI5lIhkE5pawvMmMHTSzZpnSKRDJSyQmBmD5jZVjNb1uLcQDObY2arg/sBqfp+SY3WtsDMyYpR0i+Pf/ztQj71i1d5a/PukNKJSFekskXwa2DaYeduA+a6+1hgbvBYMkhrW2B+7/KT+MutU/jP6RN4a/MePnbnX/jWk8vYtb8+7LgikgRLZVPezEYCT7v7hODxSuBcd68ys6HAc+4+rqPPKS8v94qKipTllO6zc18dP5yziodeW09RnxxumTqOT5aPIB6zsKOJRI6ZLXD38o5e19NjBCXuXhUcbwZKevj7JcUGFOTwn9Mn8IcbzmZMcV9un7WU6fe8xIL1O8OOJiJtCG2wONj/uM3miJldb2YVZlaxbdu2Hkwm3eGEYf15+Aunc+dVk9i65wCX3/syNz+yiK27W591JCLh6elCsCXoEiK4b3OjXHe/z93L3b28uLi4xwJK9zEzLp1UxryvncuXzh3N04urmPKD57nvhbXUNTSFHU9EAj1dCJ4Crg6Orwae7OHvlxAU5GZx67TjmP3Vczht1EC+86e3mHbnCzy/Si09kXSQyumjM4FXgHFmttHMrgPuAC4ws9XAR4LHEhGjBhfwwDWn8sA15TQ1OVc/8Dqf/00F7+7QfgciYUrprKHuollDvU9tQyP3v/gOd89bQ0OT84VzjmHEgHzunLuGTdU1WsxOpBskO2tIhUBCtXnXAb77zAqeXLQJ49DZA/nZcb572YkqBiJdlK7TR0UOUdo/jzuvmszgvjlHTCHTYnYiPUOFQNLCjr11rZ6vrK7huZVbaWxK/5arSKbSxjSSFoYV5VPZygqmMYNrfvVXhvXP44ryEXzilOGMGNgnhIQivZdaBJIWWlvMLj87zvcuP4l7Pn0yY0oK+cm81ZwzYz6fvf81/rB4E7UNjSGlFeld1CKQtNA8IDxj9spWZw19/KShVFbX8GjFBh6t2MgNMxcyoE820yeXceWpIziutF+Y8UUymmYNScZpbHJeWrOdhys2MGf5Fuoam5g4oogry0dw8cShFOZlhx1RJC1o+qhEwnv76nh8YSWP/HUDK7fsIT87zsdPGspVp47glKMH8OSiTW22MkR6OxUCiRR3Z9GGah6p2MBTizaxr66R4sIcdu6rp6HFjCNdmyBRousIJFLMjMlHDeC7l53E6//8Eb53xUns2t9wSBGA5msT3goppUh6UiGQXqcgN4tPlo+gvrH1FU4rqw/wtUcW89iCjWxqZcqqSNRo1pD0Wm1dm5CXHWP+yq38/o2NABw9qA9njh7EGaMHc8YxgyguzO3pqCKhUiGQXuuWqeO4fdZSaurfv96geYzgkonDWLllDy+v3cEra3fw9OIqZr6+AYCxQ/pyxuhBnDl6EB8aNYgBBTlh/RFEeoQGi6VXe2JhZVKzhhqbnOWbdvHy2h28vHYHFeveY39dI2ZwfGm/oMUwiNNGDaQwLzvpzxUJk2YNiXwAdQ1NLNlYzStBYVjw7k7qGpqIx4xhRXlUVR/QbCRJe8kWAnUNibQiJytG+ciBlI8cyA3nj+VAfSNvvLuTV9bu4OfPv93qbKR/eWIZZnBsSSHHFBeQmxVv49NF0otaBCKdNOq2Px6xZPbh4jHj6EF9GFdSyNiSQo4t6cu4kkJGDi4gO972ZD11OUl3UotAJEXamo00rH8eD1x7Kqu27GX1lj2s3LyHtzbvYfbyzTQ3ILLjxqjBBYwtKWRcUCDGlhRy9MA+PL2k6pDB7crqGm6ftRRAxUBSSoVApJPamo30jWnHcVxpvyMWwDtQ38iarXtZvXUPq7bsZdXmPSzZWM0fl1QdfE1OVoymJm+1y+l7s99SIZCUUteQSBd0RxfOvtoG1mzdy6ote1i9dS/3vfB2m68d1j+PoUX5lPbPY1j/PEr75x88N7R/HoP75hKPWUrzSubRrCGRDHPWHfNa7XLqm5vFhSeUUFV9gKpdNVTtOkBtw6FXTWfFjJJ+eQxtURwSt3xWbdnNT+ev5UCL93THLCcVl/SnQiCSYZ5YWNnmBXAtf8C6Ozv31yeKQvUBqnYfoKq6hs27DrApKBRVuw5Q19D6EhvN8rJjTJ9URlGfHAb0yWZAnxz6B/cD+mRT1CeHoj7ZrQ5uJ5u1q/8dVGC6hwaLRTJMR5vzNDMzBhbkMLAghxOG9W/1s9yd9/bVUbXrABf95MVWX3Ogvom5b22len8d9Y1t/0JYmJt1sEAUBfdzV2w5pAhAYjzjv/64ggll/SjIzaJPThYFOXGy2pkldbjDC0x3DpirwLRNLQKRXq6tLqeyonxeum0K7s7+ukZ27q+jen89O/fXsXN/PbuC+9bOr9uxP+nvz82K0Tc3iz65cQpysijIDW458YP3fXKz6JubxX0vrGVXTcMRn1FcmMvMz59OblaM3OwYuVnxxHFWDLO2x0aaZVoLprs+U11DIgKk5odgW8VlUEEO37rkBPbXNrC3toH9dY3sq21gX10D+2oTx/vrGoPngnN1DeyrbWi3VdKe5oKQm/1+ccg7eBwnNzvGq2/v4ED9kV1lfXOz+NzZo8iOGdlZMbJiRk5WjKxYjOy4kR2PkRXcH3wci5GTZWTFYvxlzTZ+MnfNIWM2eVkx/vmi47noxGHE40ZWzIhZ4j4esw4LV3f+/1IhEJGDuvu31lQUl7qGJv5mxnyqdh044rmBBTn8+yUncKC+kdqGpuDWyIH6xH1t/fvnapvPNTRRW9/EgYZGlmzc1eU/a3eLWeKCw3jMiFviPiseO1gstu2ppbGVn8vNLbjO0BiBiBw0fXJZt/YGp17KAAAGrElEQVSHJzue0Rk5WTFunXZcqwXmmxeN5+KJw7r82e11j71463k0NDn1jU3UNzoNwX3icRMNTU5dQ+K+obGJusYmGoLnr3uw7V9Qv3XxeBqbnMbg+pDm60Sa/NDHh7/m4YoNrX5eKvfOUCEQkS7p7uLS/JnQvQUG2r4I8Jap4zCzg90+nVXWxlXmZUX5XHvWqC5lfXHN9tavXC/K79LnJUOFQETSSm8pMOn0mR1RIRCRSMiUApOqotUeDRaLiPRSyQ4Wa/N6EZGIUyEQEYk4FQIRkYhTIRARiTgVAhGRiMuIWUNmtg1YH3aOwwwGtocdIkmZlBUyK28mZYXMyptJWSE98x7t7sUdvSgjCkE6MrOKZKZlpYNMygqZlTeTskJm5c2krJB5eVtS15CISMSpEIiIRJwKQdfdF3aATsikrJBZeTMpK2RW3kzKCpmX9yCNEYiIRJxaBCIiEadC0AlmNsLM5pvZm2a23MxuCjtTMswsbmYLzezpsLO0x8yKzOwxM3vLzFaY2RlhZ2qPmX01+HuwzMxmmlle2JlaMrMHzGyrmS1rcW6gmc0xs9XB/YAwMzZrI+uM4O/CEjN73MyKwszYrLWsLZ77mpm5mQ0OI1tXqRB0TgPwNXcfD5wOfNnMxoecKRk3ASvCDpGEO4Fn3f04YCJpnNnMyoAbgXJ3nwDEgavCTXWEXwPTDjt3GzDX3ccCc4PH6eDXHJl1DjDB3U8CVgG393SoNvyaI7NiZiOAC4F3ezrQB6VC0AnuXuXubwTHe0j8oErdIuHdwMyGAx8Hfhl2lvaYWX/gHOB+AHevc/fqcFN1KAvIN7MsoA+wKeQ8h3D3F4D3Djt9KfBgcPwgML1HQ7Whtazu/md3bwgevgoM7/FgrWjjvyvAj4BvABk38KpC0EVmNhKYDLwWbpIO/ZjEX86msIN0YBSwDfhV0I31SzMrCDtUW9y9Evg+id/+qoBd7v7ncFMlpcTdq4LjzUBJmGE64XPAM2GHaIuZXQpUuvvisLN0hQpBF5hZX+D3wFfcfXfYedpiZhcBW919QdhZkpAFnAzc6+6TgX2kT7fFEYK+9UtJFLBhQIGZ/V24qTrHE1MG0/63VzP7ZxLdsg+FnaU1ZtYH+Cfgm2Fn6SoVgk4ys2wSReAhd58Vdp4OnAVcYmbrgN8BU8zsf8ON1KaNwEZ3b25hPUaiMKSrjwDvuPs2d68HZgFnhpwpGVvMbChAcL815DztMrNrgIuAz3j6znUfTeIXgsXBv7XhwBtmVhpqqk5QIegEMzMSfdgr3P2HYefpiLvf7u7D3X0kiYHMee6elr+1uvtmYIOZNe/QfT7wZoiROvIucLqZ9Qn+XpxPGg9ut/AUcHVwfDXwZIhZ2mVm00h0a17i7vvDztMWd1/q7kPcfWTwb20jcHLwdzojqBB0zlnAZ0n8Zr0ouH0s7FC9yA3AQ2a2BJgEfCfkPG0KWi6PAW8AS0n8W0qrK0vNbCbwCjDOzDaa2XXAHcAFZraaRKvmjjAzNmsj691AITAn+Lf2s1BDBtrImtF0ZbGISMSpRSAiEnEqBCIiEadCICIScSoEIiIRp0IgIhJxKgQSKWbW2GLq7yIz67arl81sZGsrUoqku6ywA4j0sBp3nxR2CJF0ohaBCGBm68zse2a21MxeN7MxwfmRZjYvWBN/rpkdFZwvCdbIXxzcmpeXiJvZL4J9Cv5sZvnB60eb2bNmtsDM/mJmxwXnPxHsZ7DYzF4I5Q8vkadCIFGTf1jX0JUtntvl7ieSuKL1x8G5nwAPBmviPwTcFZy/C3je3SeSWBNpeXB+LHCPu58AVAOXB+fvA25w91OArwM/Dc5/E5gafM4l3f2HFUmGriyWSDGzve7et5Xz64Ap7v52sLDgZncfZGbbgaHuXh+cr3L3wWa2DRju7rUtPmMkMCfY9AUzuxXIJlFUtgErW3xlrrsfHyybMBp4BJjl7jtS8McWaZfGCETe520cd0Zti+NGIJ9Ey7u6tbEJd/+imX2IxOZBC8zsFBUD6WnqGhJ535Ut7l8Jjl/m/S0oPwP8JTieC3wJDu4J3b+tDw32rHjHzD4RvN7MbGJwPNrdX3P3b5JoNYzoxj+PSFJUCCRqDh8jaLn65oBg5dObgK8G524Arg3OfzZ4juD+PDNbCiwAOtq7+jPAdWa2mMR4wqXB+RnBAPUyEkUnI3e4ksymMQIRDo4RlLv79rCziPQ0tQhERCJOLQIRkYhTi0BEJOJUCEREIk6FQEQk4lQIREQiToVARCTiVAhERCLu/wO4mSf2No0ylgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f51502c1750>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_decision_region(X_std, y, classifier=ada)\n",
    "\n",
    "plt.title(\"Adaline - Gradient Descent\")\n",
    "plt.xlabel(\"sepal length [standardized]\")\n",
    "plt.ylabel(\"petal length [standardized]\")\n",
    "plt.legend(loc=\"upper left\")\n",
    "plt.show()\n",
    "\n",
    "plt.plot(range(1, len(ada.cost_) + 1), ada.cost_, marker='o')\n",
    "plt.xlabel(\"Epoches\")\n",
    "plt.ylabel(\"Sum-Squared-Error\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wow 标准化后的数据再使用梯度下降Adaline算法竟然收敛了！ 注意看Sum-squared-error\\(即，$y-w^{T}x$\\)最后并没有等于0，即使所有样本都正确分类。\n",
    "\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 大规模机器学习和随机梯度下降\n",
    "\n",
    "在上一节我们学习了如何使用梯度下降法最小化损失函数，由于梯度下降要用到所有的训练样本，因此也被称为批梯度下降\\(batch gradient descent\\)。现在想象一下我们有一个非常大的数据集，里面有几百万条样本，现在用梯度下降法来训练模型，可以想象计算量将是非常大，每一次求梯度都要用到所有的样本。能不能用少量的样本来求梯度呢？\n",
    "\n",
    "随机梯度下降法\\(stochastic gradient descent\\)诞生啦！有时也被称为迭代\\(iteration\\)/在线\\(on-line\\)梯度下降。随机梯度下降法每次只用一个样本对权重进行更新\\(唔，感知机算法也如此，转了一圈，历史又回到了起点。\\)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\\eta(y^{(i)}-\\phi(z^{(i)}))x^{(i)}$$\n",
    "\n",
    "\n",
    "虽然随机梯度下降被当作是梯度下降的近似算法，但实际上她往往比梯度下降收敛更快，因为相同时间内它对权重更新的更频繁。由于单个样本得到的损失函数相对于用整个训练集得到的损失函数具有随机性，反而会有助于随机梯度下降算法避免陷入局部最小点。在实际应用随机梯度下降法时，为了得到准确结果，一定要以随机方式选择样本计算梯度，通常的做法在每一轮迭代后将训练集进行打乱重排\\(shuffle\\)。\n",
    "\n",
    "**Notes:**在随机梯度下降法中，通常用不断减小的自适应学习率替代固定学习率$\\eta$,比如$\\eta = \\frac{c_{1}}{[number of iterations] + c_{2}}$,其中$c_{1},c_{2}$是常数。还要注意随机梯度下降并不能够保证使损失函数达到全局最小点，但结果会很接近全局最小。\n",
    "\n",
    "随机梯度下降法的另一个优点是可以用于在线学习\\(online learning\\)。在线学习在解决不断累积的大规模数据时非常有用，比如，移动端的顾客数据。使用在线学习，系统可以实时更新并且如果存储空间快装不下数据了，可以将时间最久的数据删除。\n",
    "\n",
    "**Notes** 除了梯度下降算法和随机梯度下降算法之外，还有一种常用的二者折中的算法：最小批学习\\(mini-batch learning\\)。很好理解，梯度下降每一次用全部训练集计算梯度更新权重，随机梯度法每一次用一个训练样本计算梯度更新权重，最小批学习每次用部分训练样本计算梯度更新权重，比如32。相对于梯度下降，最小批收敛速度也更快因为权重参数更新更加频繁。此外，最小批相对于随机梯度中，使用向量操作替代for循环，使得计算更快。\n",
    "\n",
    "上一节我们已经实现了梯度下降求解Adaline，只需要做部分修改就能得到随机梯度下降法求解Adaline。第一个修改是fit方法内用每一个训练样本更新权重参数$w$,第二个修改是增加partial\\_fit方法，第三个修改是增加shuffle方法打乱训练集顺序。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "from numpy.random import seed\n",
    "\n",
    "class AdalineSGD(object):\n",
    "    \"\"\"ADAptive LInear NEuron classifier.\n",
    "\n",
    "    Parameters\n",
    "    ---------------\n",
    "    eta: float\n",
    "        Learning rate (between 0.0 and 1.0)\n",
    "    n_iter: int\n",
    "        Passes over the training dataset.\n",
    "\n",
    "    Attributes\n",
    "    ---------------\n",
    "    w_: 1d-array\n",
    "        Weights after fitting.\n",
    "    errors_: list\n",
    "        Number of misclassification in every epoch.\n",
    "    shuffle: bool (default: True)\n",
    "        Shuffles training data every epoch\n",
    "        if True to prevent cycles.\n",
    "    random_state: int (default: None)\n",
    "        Set random state for shuffling\n",
    "        and initializing the weights.\n",
    "\n",
    "    \"\"\"\n",
    "    def __init__(self, eta=0.01, n_iter=10, shuffle=True, random_state=None):\n",
    "        self.eta = eta\n",
    "        self.n_iter = n_iter\n",
    "        self.w_initialized = False\n",
    "        self.shuffle = shuffle\n",
    "        if random_state:\n",
    "            seed(random_state)\n",
    "\n",
    "    def fit(self, X, y):\n",
    "        \"\"\"Fit training data.\n",
    "\n",
    "        Parameters\n",
    "        -----------\n",
    "        X: {array-like}, shape=[n_samples, n_features]\n",
    "        y: array-like, shape=[n_samples]\n",
    "\n",
    "        Returns\n",
    "        --------------\n",
    "        self: object\n",
    "        \"\"\"\n",
    "        self._initialize_weights(X.shape[1])\n",
    "        self.cost_ = []\n",
    "        for i in range(self.n_iter):\n",
    "            if self.shuffle:\n",
    "                X, y = self._shuffle(X, y)\n",
    "            cost = []\n",
    "            for xi, target in zip(X, y):\n",
    "                cost.append(self._update_weights(xi, target))\n",
    "            avg_cost = sum(cost)/len(y)\n",
    "            self.cost_.append(avg_cost)\n",
    "        return self\n",
    "\n",
    "    def partial_fit(self, X, y):\n",
    "        \"\"\"Fit training data without reinitializing the weights.\"\"\"\n",
    "        if not self.w_initialized:\n",
    "            self._initialize_weights(X.shape[1])\n",
    "        if y.ravel().shape[0] > 1:\n",
    "            for xi, target in zip(X, y):\n",
    "                self._update_weights(xi, target)\n",
    "        else:\n",
    "            self._update_weights(X, y)\n",
    "        return self\n",
    "\n",
    "    def _shuffle(self, X, y):\n",
    "        \"\"\"Shuffle training data\"\"\"\n",
    "        r = np.random.permutation(len(y))\n",
    "        return X[r], y[r]\n",
    "\n",
    "    def _initialize_weights(self, m):\n",
    "        \"\"\"Initialize weights to zeros\"\"\"\n",
    "        self.w_ = np.zeros(1 + m)\n",
    "        self.w_initialized = True\n",
    "\n",
    "    def _update_weights(self, xi, target):\n",
    "        \"\"\"Apply Adaline learning rule to update the weights\"\"\"\n",
    "        output = self.net_input(xi)\n",
    "        error = (target - output)\n",
    "        self.w_[1:] += self.eta * xi.dot(error)\n",
    "        self.w_[0] += self.eta * error\n",
    "        cost = 0.5 * error ** 2\n",
    "        return cost\n",
    "\n",
    "    def net_input(self, X):\n",
    "        \"\"\"Calculate net input\"\"\"\n",
    "        return np.dot(X, self.w_[1:]) + self.w_[0]\n",
    "\n",
    "    def activation(self, X):\n",
    "        \"\"\"Compute linear activation\"\"\"\n",
    "        return self.net_input(X)\n",
    "\n",
    "    def predict(self, X):\n",
    "        \"\"\"Return class label after unit step\"\"\"\n",
    "        return np.where(self.activation(X) >= 0.0, 1, -1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\\_shuffle方法的工作方式：调用numpy.random中的permutation函数得到0-100的一个随机序列，然后这个序列作为特征矩阵和类别向量的下标，就起到了shuffle的功能。\n",
    "\n",
    "我们使用fit方法训练AdalineSGD模型，使用plot\\_decision\\_regions对训练结果画图:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<__main__.AdalineSGD at 0x7f51504529d0>"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ada = AdalineSGD(n_iter=150, eta=0.01, random_state=1)\n",
    "ada.fit(X_std, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEWCAYAAABmE+CbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3XmYHVW1/vHvm0ECpiEQAjKEBLgBjKiIYRIIIKjgdUJF8XrFCArihNNVAR8FB+LsdQS5RBFBUX+IIiKjQAAVkiDIHJEpDDJJIDEGCFm/P/Y+9OnOGar7zN3v53n66VNV51Stqk5qnVq19y5FBGZmZmM6HYCZmXUHJwQzMwOcEMzMLHNCMDMzwAnBzMwyJwQzMwOcEHqSpDmSriz43uMknZ5fbyFpuaSxrY2wcySdKukLbdjO2yVd2OrtDFf5cZC0p6TbOh2TdT8nhC4i6TJJj0laqxXrj4h7ImJiRDzTivXXImlzSWdJekTS45JulDQnL5suKSSNa3dcRVSKLyLOiIhXDnN9sySdm//WSyXdLOmLktZvXtT9IuKKiNi2GeuSdJek/Wos31vS6vzFY7mkeyX9QtJOzdh+K+T/d+/udBzdwAmhS0iaDuwJBPC6jgbTGj8BlgDTgMnAO4AHOxpRB0h6GXAZcBWwXURMAvYHVgEvrvKZrkyUNdwfEROBPmBX4FbgCkn7djYsq8cJoXscAvwZOBV4Z/kCSZMlnSPpCUnXAFsPWv4tSUvy8kWS9qy0gcHfdPM3o89LukrSMkkXStqw7P27Svpj/hZ7vaS9G9i/nYBTI+JfEbEqIv4SEb/Py+bn30vzt8rdJI2R9GlJd0t6SNJpktYri22PstiWlK42svUl/S7v09WSti77XNVjJWlnSQvzsgclfaNGfAPKdpJeIOkiSf/Mnz2mynH4CvCjiJgbEQ/Cs1dun42Iy/K65uS/yTclPQocJ2lrSX+Q9Gi+yjpD0qSy7b9E0rV5n38OTChbtreke8umN81Xaw9LulPSh8qWHZe/0Z+W13WTpFl52U+ALYDf5uPwiSr7SN6viIh7I+IzwCnAl8u2s13Z8bpN0lvKlr06XzUtk3SfpI+XLXu9pOvy3+jvkvbP89eTNE/SA/kzX1AujZb+VpK+pnRVdqekA/KyL5K+iH0379N3a+3TiBcR/umCH+B24H3AS4GngY3Llp0J/AJ4LrA9cB9wZdny/yZ96x4HfAz4BzAhLzsOOD2/nk66AhmXpy8D/g5sA6ydp7+Ul20GPAq8mvTF4RV5esow9+9i0rfig4EtBi0bEFeed2g+JlsBE4FfAT/Jy6YBy4C3AePzvu+Ql52a49w5H48zgDMLHqs/Ae/IrycCu9aIb07pb0D6JvxAXt+EPL1LhWPwXOAZYO86x2oO6YrhgznOtYH/yH+DtYAppCT1v/n9zwHuBj6Sj8eb87+hL+TlewP35tdjgEXAZ/LntgLuAF5V9u9lZf67jwXmAn8ui+0uYL8asT+7rUHzXw6szsfguaSrxXfl/XsJ8AgwM7/3AWDP/Hp9YMf8emfg8XwcxpD+jW6Xl50N/CCveyPgGuCIsuP5NPCevE9HAvcDKvt/8O5OnwO64afjAfgnAPbI/2A3zNO3Ah/Jr8fmZduVvf8EyhJChfU9Brw4vz6O2gnh02Wfex9wfn79SfIJuGz5BcA7h7mP6wNfAm4inRSvA3aqFFeedwnwvrLpbfNxGAccDZxdZTunAqeUTb8auLXgsZoPHF/6O5S9p1J8c+hPCG8D/lLgGGye11P+t/wKsBT4V+lvkdd9T511vaG0TWB2+Qkuz/sjlRPCLoPXnY/nj8r+vVxctmwm8O+y6bsYXkLYLu/7ZsBbgSsGLf8B8Nn8+h7gCGDdCu/5ZoV1bww8CaxdNu9twKVlx/P2smXr5FieV/b/wAkhwiWjLvFO4MKIeCRP/5T+stEU0klwSdn77y7/sKSPS7pF6WbtUmA9YEOK+UfZ6xWkb8aQvoUflEsyS/N69wA2GbwCpRY3pZuIvx+8HCAiHouIT0XEC0j/ga8Dfi1JVeLadNB+3k06DhsDU0lXNkPdp3rH6jDS1dKtkhZIek2NbZSrF0/JY6Rvyc8ew4j4RKT7CGeT9q+k/O+NpI0lnZnLIU8Ap5fFvSlwX+SzWzbg30iZacCmg/6ux5COa8ng4zdBjd/H2Ix0El6aY9hlUAxvB56X3/smUiK/W9LlknbL86sd52mkK6MHytb3A9KVwhr7FBEr8suJ2AC9drNqxJG0NvAWYKyk0j/atYBJkl4M3EgqH0wlXTlAquOWPr8n8AlgX+CmiFgt6TGg2om2qCWkK4T31HtjRJxBKs0UEhGPSPoaKeltQDpRDHY/6T96yRak4/Bgjm3notsrqXesIuJvwNskjQHeCPw/SZOrxFduCakUVlNE/EvS1Xndl9Z7+6DpE/K8F0bEPyW9ASjVux8ANpOksqSwBZVPnkuAOyNiRr14C8ZV1IHAtfkYLAEuj4hXVNxAxALg9ZLGAx8glUunkmLfusJHlpCuEDaMiFXDiM1DPme+Qui8N5BKKDOBHfLP84ErgEMiNRH9FenG4jqSZjLwpnMf6UT5MDBO0meAdZsQ1+nAayW9StJYSRPyzcnNh7MySV+WtL2kcZL6SHXc2yPi0Rz7alI9u+RnwEckbSlpIumE+PP8H/4MYD9Jb8nrmyxphwJh1DxWkv5b0pSIWE36JkuOq1J85c4FNpH0YUlrSeqTtEuV934COFTSpyRtlLe7ObBlgdiXA49L2gz4n7Jlf8r79SFJ4yW9keoJ8xpgmaRPSlo7/223V/FmoQ9S/TgMoGQzSZ8F3k26EoF0vLaR9I4c73hJO0l6vqTn5CvO9SLiaeAJ0rEHmAe8S9K+So0ONpO0XUQ8AFwIfF3SunnZ1pL2avY+jXROCJ33TlL99p6I+Efph/Tt7+35Uv0DpMvbf5Bq5D8q+/wFwPnAYlKZYCWDyg3DERFLgNeT/hM/nNf5Pwz/38w6pLLIUtJNzGnk5rX5Ev6LwFX5kn9X4IekpqrzgTtJ+/XB/P57SCWFjwH/JJWfKjbZHKTesdofuEnScuBbwMER8e8q8T0rIpaRbnS+lvQ3+huwT6UAIuJK0g3W2cDiXN44n1TH/k6N2I8HdiTdVP0d6UtCaZ1Pka465pCOx1vLlw/a/jPAa0hfPO4k3cw9hVQ6K2Iu8Ol8HD5e5T2b5mO4HFgAvJB0I/3CHMMy4JWkq6r7Scfsy6QrY0hNku/KpbH3kspJRMQ1pBvR38zH4XL6ryIPId0kv5lUmvt/VChvVvEt4M1KLZC+XfAzI5IifLVkZma+QjAzs8wJwczMACcEMzPLnBDMzAzosX4IEyduGJMnT+90GGZmPeWeexY9EhFT6r2vpxLC5MnTOfbYhZ0Ow8yspxxxhKr1XB/AJSMzMwOcEMzMLHNCMDMzoMfuIVQybtzTbLXVvayzzspOh1LYihUTuOOOzVm1anynQzEze1bPJ4SttrqXqVP76OubTvWRlLtHRLBs2aPAvSxeXG88MzOz9un5ktE666ykr29yTyQDAEn09U3uqSsaMxsdej4hAD2TDEp6LV4zGx1GREIwM7PGOSE00d/+disHHLAbm2++Ft/73tc6HY6Z2ZD0/E3lbjJp0gaccMK3+f3vf93pUMzMhmxUJYRJr9yJMY88tMb81RtuxNILFzS8/ilTNmLKlI246KLfNbwuM7N2G1UJYcwjD7F68prjO1VKEmZmo43vIZiZGeCE0LB5877HPvvswD777MA//nF/p8MxMxu2UVUyaoXDDns/hx32/k6HYWbWMCeEJnrwwX/wylfOYtmyJxgzZgwnn/y/XHnlzfT1rdvp0MzM6hpVCWH1hhtVbWXUDBtv/Dyuv/7epqzLzKzdRlVCaEbTUjOzkco3lc3MDHBCMDOzrGMJQdJUSZdKulnSTZKO6lQsZmbW2XsIq4CPRcS1kvqARZIuioibOxiTmdmo1bErhIh4ICKuza+XAbcAm3UqHjOz0a4r7iFImg68BLi6wrLDJS2UtHD58ofbHVohRx11KDNnbsTs2dt3OhQzs2HreEKQNBE4C/hwRDwxeHlEnBwRsyJi1sSJaw5MN1QRtaeH4+CD53Dmmec3viIzsw7qaEKQNJ6UDM6IiF+1enunnQYnntifBCLS9GmnNbbe3XabzaRJGzQeoJlZB3WylZGAecAtEfGNVm8vApYvh7PP7k8KJ56Yppcvb86VgplZL+tkK6PdgXcAN0i6Ls87JiLOa8XGJDjyyPT67LPTD8CBB6b5fu69mY12HUsIEXEl0NbTcCkplJIBOBmYmZV0/KZyO5XKROXK7ymYmY1moyYhlN8zOPBAuPDC9Lv8nsJwHXHE23j1q3fj9ttv48Uv3pwzzpjXvMDNzNpk1Ix2KsHEiQPvGZTuKUyc2FjZ6Ac/+FlzgjQz66BRkxAADjkkXQmUTv6lpOB7CGZmo6hkVDL45O9kYGaWjIiEED12V7jX4jWz0aHnE8KKFRNYtuzRnjnJRgTLlj3KihUTOh2KmdkAPX8P4Y47NgfuZZ11unPgu0pWrJiQ4zYz6x49nxBWrRrP4sVbdjoMM7Oe1/MlIzMzaw4nBLMe1Iph3M2cEMx6zPz5cPHFA4dxv/jiNN+sEVXvIUg6p8Dn/xkRc5oXjpnVEgFPPgnXXJOm99svJYNrroGddx7Y8dJsqGrdVH4+8O4aywV8r7nhmFktUkoCkJJAKTHsvHOaPxKTweAk56TXOrUSwrERcXmtD0s6vsnxmFkdpaRQSgYwcpPB/Pnpiqi0f6Xy2FprwezZnY5u5Kl6DyEiflHvw0XeY2bNVTopliu/pzBSlJfHSvtXKo89+eTI299uUOsewm+Bqoc8Il7XkojMrKryk2KpTFSahuZeKXS6VNPM8lin96VX1CoZfS3/fiPwPOD0PP024MFWBmVmlUmpXFJ+UiydNNdaq3knuW4p1TSjPNYt+9ILqiaE0v0DSV+PiFlli34raWHLIzOzimbPXnMY92ZfGXRLS6Zq5bGi+9tN+9ILigxd8VxJW0XEHQCStgSe29qwzEa3eiWOVg7j3i0tmYZSHqt2vLplX3pFkY5pHwEuk3SZpMuBS4EPtzYss9GrGzqelZ9IS9p9Aq1WHtt554HlsXrHqxv2pVfUvUKIiPMlzQC2y7NujYgnWxuW2ejULSWORks1zVKvPFbkeJViL9eJfekFdROCpHWAjwLTIuI9kmZI2jYizm19eGajSzeUONrZkqmIWuWxescLumtful2RktGPgKeA3fL0fcAXWhaR2SjXrBLH6tW1p2ttf621YKedBpZqdtqpuS2Ziqo3kF+t41W07OTBApMiCWHriPgK8DRARKwgDVthZi3QjI5n8+bBiSf2J4HVq9P0vHnNi7MditxPqXe8Zs8emFBLSaHU5LQb7tl0iyIJ4SlJa5M7qUnaGvA9BLMWGFyuOeaY9Lu8t249q1fDypVw6639SeHEE9P0ypX1rxRKdfkFCwb2EF6woL09hIv0VC56vKqVndwbeqAizU6PA84Hpko6A9gdmNPCmMxGrWZ0PBszBo48Er7//ZQEjjoqzd922zR/TJ2vge3uIdxok9FGjlc33LPpJkVaGV0oaRGwK6lUdFREPNLyyMxGqWZ0PLvySthmm5QQSr1zt9kmzS/SO7ddPYTrvadIHI0er9E0WGA9dUtGki4BdomI30XEuRHxiKST2xCb2ajVSMeziFQaOu+8VB6KSL/POy/NL1IGafQ+RtFyT9GSUL04Gj1eo2GwwCKKlIy2BD4paaeIKA13PavWB8y6VaODnDVSAmmm1asHln7KpyNg8WJ46il4znNgyhR4+OE0vXjxwG/jleJsZw/hTjcZ7bYmtp1W5KbyUmBfYGNJv5W0XotjMmuJRluTFPl8O1qs1GtBNGYMrL02zJyZkgGk3zNnpvljxtSOs509hJvRZLQR7dhGLymSEBQRqyLifcBZwJXARq0Ny6y5Gm1N0qwSSKOKtiA69NB0z6DcNtuk+UXirNdUs1nlnkabjDZDO7bRK4qUjE4qvYiIUyXdALy/dSGZNV/REkYrSyDN+LZZakFUSgKlFkTbbdffgqi8mejgMkhpPxqNsxk9hEvvufpq2GWX/vdcfXX/e9r1Db2VgwX2kqpXCJLWzS9/KWmD0g9wJ/DxtkRn1kT1ShitLoE0SykplCtvTlqkDNLosai3r0VjuOceWH992HffNL3vvmn6nnvStDuNtVetktFP8+9FwML8e1HZtFlPqVWeaFcJpBlKZaJy5fcUoFjJp5FjUWRfi8SwxRawdClcckmavuSSNL3FFml/3GmsvaqWjCLiNfn3lu0Lx6w1irQmaVYJpJUtVsrvGZTKRKXpE09c80qhXHlZrF6ppl5Jqcg6SlcBlWIova63HXcaa69aJaMda/00Y+OSfijpIUk3NmN9ZtU0WkYp+vlWt1gZMwYmTBh4z+DII9P0hAn1eyGXjkW9Uk29klKRdRRRZDutLsFZv1o3lb+ef08g9Tu4HhDwIlLJaLcqnxuKU4HvAqc1YV1mNRUZW7/WuPlFesS2+vGWAIcdNrDfQSkpFEkG0F+queaaVKLZb7/+Uk3RZwgUWUeRfa53zOstt+aqVTLaB0DSr4AdI+KGPL09aXyjhkXEfEnTm7EusyLqlVGaUe5pRse0eusYfPIvmgygOS2EmlHOqXfM9903JRl3GmufIs1Oty0lA4CIuFHS81sY0wCSDgcOB9hggy3atVkbZaqVe6C/3FNkbJ5581J/gNI39lLNf8KE9M2+iCLbacb+1hq/p8iAcfXWUSSGWtsZM6bxgf5saIokhBsknQKcnqffDvy1dSENFBEnAycDTJs2y+0KrGVqlXvKW97AwG+rpRJJaQyh8hu85TeABw83UUmR7TTjRNiM8lgzyjn1ttOOEpz1K5IQ5gBHArkLDPOBE6u+26yHVSsplU5EEQNLJOVPFZOKDTtdqxzUjpY1RctjtVoINbPEVq9TmDuNtU/N7yuSxgLzIuKbEXFg/vlmRKxsU3xmXeOKK+rPLw07Xd5WvzTsNDTe4asZmtEaqh0tqqz9aiaEiHgGmCbpOa3YuKSfAX8CtpV0r6SCVVaz9iqVgy65BJYtS/OWLUvTpSGlS++pNux00Y5W7ejc1ozxezwG0MhTpGR0B3CVpHOAf5VmRsQ3Gt14RLyt0XWYNVOtYaWh/35C6WdwGaXesNP1yk5FSzHNGIa7GaUYl3NGliKN1f4OnJvf21f2Yzai1BpWWkothV7+cujrS9N9fWl6woQ0XWTY6VJ5qfxqAPrnFynFdMsw3Dby1L1CKHsojtmINXhY6UothPbcEy66aGCJBNL8kkMPTe9ZsKB/3jbbwCte0V9S+sMf0uu+vlR2+sMfUmIpfYNvtLUTtKelko08dROCpCnAJ4AXkHotAxARL29hXGZraEaHr2rqDSst1R9SGmq/Z99903tKZahSS57yQelK6rV2gs4Ow20jU5GS0RnArcCWwPHAXcCCWh8wa7Z2lEBqDSstpTF6Jk0aOHbPpEkDx/+pVe4ZMwaWLElDPvTlomtfX5pesmRozTS7YRhuG3mKJITJETEPeDoiLo+IQwFfHVjbtONJZFB7WOnS2D2PPTZwqObHHkvziwz5HAFTp6aTf3lLpSVL0vyi+9Etw3DbyFOkldHT+fcDkv4TuB/YoHUhmQ3Ujs5aRYaVrhRDaejnobTeqdVSqZ4irZDAD4634SlyhfAFSesBHyM9Ke0U4CMtjcpskFaXQErDSpf3Kj7yyDRdGla60RiKtFQqso5uGIbbRqYirYzOzS8fB/ZpbThmlbVjGOQZM+Df/x5Y7pkxIzUZbVYMRVoq1dMtw3DbyFM1IUj6DlC14hgRH2pJRGaDNHPcnFrbePJJWLgwXQ2UtrFwYdrm6tWND8Vc2o9aLZWaNf5P0feYlat1hVB6bvLuwEzg53n6IODmVgZlVq5aCQSaVwIpMnhdozG0Yz/MGqGo0+xA0p+BPSJiVZ4eD1wREbu2Ib4Bpk2bFcceu7D+G21EamU/BEhNWFeu7P/GHpFO3hMmDHwwfKMxtHo/zAY74ggtiohZ9d5X5Kby+sC6ZdMT8zyztmplCaS8F/GyZWm61Iu4NHhds2JwKce6VZFmp18C/iLpUkDAbJr0CE2zblP6tl76GXwB7W/3NpIVaWX0I0m/B3bJsz4ZEf9obVhm7VVqErrvvv3jEPX1pXsIpSah7Xi0pVknFblCABgLPJzfv42kbSLC4ybaiFJqElppfrsebWnWDPPnw1W/vA+eeQbGji38uSKD230ZeCtwE1AahitIj9I0K6TbSy1FmoQW7S3d7ftqI8fcD+aTfgVHTzqJw0+YDoCOKLa+IlcIbwC2jYgni63SbKBeKLUUbRK63379yaA0XX6y74V9td4ydy6w5J6qy+/8/vlVlkwf8raKPjFtPOCEYEPWS6WWer176/VU7qV9tc6oNjrvVWdWP+ED3Dl1Lzj66BZENFCRhLACuE7SJZQlBfdUtiLaMTBdM1VrElq0t3Qv7au1xrx5lec/9PcnYOlSNhr76BrLdp9wF6d/4+Eaa219MoBiCeGc/GM2LKUTZa1SS7crWlIaCftq9c2dW2XBknvYfcKiiotmAKcffFZX1w6LNDv9cTsCsZGrHQPTtUORAeNGyr5a+qb/0N+fWHPBsmXwzDMcPemkNZdN4tkbuZV1bzKAYq2MZgBzSeMZlT9Cc6sWxmU9pFarmtIJ8uqr+58dUJqG3jtR1upl3I5B+Ky55n70weoLVz6ZWur0/Wzg/D5yPX96CyPrjCIlox8BnwW+SRr++l0UG/LCRoF6rWpKj55cf/2Bj55cvLj/0ZMjhQev605z5wL337fmgtxc886XHlT9w4cdRrvq992gSEJYOyIukaSIuBs4TtIi4DMtjs26XJFWNZAeMXnNNWn46P32S7+XLh2ZLW/8HILOqdUm/86XHpQebjHY7NnAYa0NrIcUSQhPShoD/E3SB4D7SAPc2ShXtFXNaGt548HrWmPuRx+EldVbv+8+YRGnf6daSx2f9IsokhCOAtYBPgR8nlQ2OqSVQVnvKNKqxi1vrKh58+ChRcPphGXNUCQhTI+IBcBy0v0DJB0EXN3KwKw3FGlV45Y3o9NwOmFtNPZRjp70yzotdaxViiSEo4FfFphno0yRVjXgljcjXaWOWI11wpretNhsaGo9U/kA4NXAZpK+XbZoXWBVqwOz7le0VY1b3vS2qp2wAJbcw0ZjH2XG+LsGzJ4BnP7SL+VWOtYral0h3E96rvLrgPKud8uAj7QyKOsdRVrVuOVN95s/H646v0InrJX/frY9fkU1O2I5GfSaqgkhIq4Hrpf004h4GkDS+sDUiHisXQFa9yvSqsYtb7pD1Y5YK59MrXSmfHTg/BHcCcvWVOQewkWSXpffuwh4SNIfI8JXCdY0foZAc1TthAX1O2KNsk5YtqYiCWG9iHhC0ruB0yLis5L+2urAbPTwMwSGbu77ajTNnLoX7L575YXuiGU1FEkI4yRtArwFOLbF8dgo42cIVDb3mNRKp5qNxj7K1d9ZUGWpv+Xb8BRJCJ8DLgCujIgFkrYC/tbasGy0GC3PEBhOm/w7Dz7Gl0jWVkWGv/4lZX0OIuIO4E2tDMpGl5HQk7naCR/6H3ZeqU1+7U5YTgbWXrX6IRweESfX+nCR99T5/P7At4CxwCkR8aXhrst6Vy/1ZK7YCeshaj4YZffx1BhjZ3qzQjNrWK0rhE9JeqTGcpHGORpWQpA0Fvge8ArgXmCBpHMi4ubhrM96U7c9Q6BeJyxgjRO/O2HZSFErIVwOvLbO5y9qYNs7A7fnEhSSzgReDzghjCKdeobA3GPcCctssFod097V4m1vBiwpm74X2GXwmyQdDhwOsMEGW7Q4JOuEVvVkHlYnrN13941cG7WKtDLqqHyP4mSAadNmRYfDsRYZTk/mefPgoeuqdMICeOYZd8IyG4JOJoT7gKll05vneWYD1HoS1tGTTuLw/as03XQnLLMh6WRCWADMkLQlKREcDPxXB+OxDpo7l2dv2lZS/cEo03FLHbPmqJsQJK1F6ncwvfz9EfG5RjYcEavyIzkvIDU7/WFE3NTIOq07DKcTFuQxdtxSx6xjilwh/AZ4nDSwXfUHmg5DRJwHnNfMdVrrzZ8Pf6vSV/2hm9Jzb4f3YBQnA7NOKpIQNo+I/VseiXWdefNyp6vBanTCmgGcfvBZbqlj1oOKJIQ/SnphRNzQ8mis7Wqd9IGKbfIPn/qzPEZ+NU4GZr2o1tAVNwCR3/MuSXeQSkYCIiJe1J4QrVEVO2GVLF1a56Q/vcKH3FzTbCSqdYXwmrZFYQ2be8wTqaftYLmef/Wmb6z8wa03qnIj1yd9s9GmVk/luwEk/SQi3lG+TNJPgHdU/KC1xPz5edTMatwJy8waVOQewgvKJ/KgdC9tTTjmTlhm1im17iEcDRwDrC2pVIQW8BTDHOHUEnfCMrNuVKtkNBeYK2luRLjeUMWwO2FN3atOSx0zs/YqUjL6paQdB817HLg7Ila1IKauVPHBKH9Pz70dXicsJwMz6y5FEsL3gR2Bv5JKRi8EbgTWk3RkRFzYwvjapt6DUTYa+ygzxt81YLY7YZnZSFIkIdwPHFYaZ0jSTOBzwCeAXwE9kxDmzcvf6gdbtgyeeWaYD0ZxMjCzkaFIQtimfNC5iLhZ0nYRcYe67YG3FOuEdXjfzwbO76NGJywzs9GhSEK4SdKJwJl5+q3AzXkU1KdbFlkNc+cCD1d4GtbKNPbenVP3qvzBZzthuX5vZjaYImo/hEzS2sD7gD3yrKtI9xVWAutExPKWRlhmrTEvjk3Hn1egE5aZmZXoiCMWRcSsuu+rlxC6yazJk2PhAQf4Jq6Z2RAUTQhFHpCzO3AcMI2BD8jZqpEAh2XiRCcDM7MWKXIPYR7wEdIDciqPqWBmZj2vSEJ4PCJ+3/JIzMyso4okhEslfZXU5+DZR2hGxLUti8rMzNquSELYJf8uvyERwMubH46ZmXVK3YQQEfu0IxAzM+usIq2MNgZOADaNiAPy0BW7RUSF4d5sxDnhhDS0x2B9fXDtuixNAAAOVUlEQVTMMb2zDTOra0yB95wKXABsmqcXAx9uVUDWZZYtS819B/9UOoF38zbMrK4iCWHDiPgFsBogD3nt5qdmZiNMkYTwL0mTSTeSkbQr6XkIZmY2ghRpZfRR4Bxga0lXAVOAN7c0KjMza7sirYyulbQXsC3pATm3RURHRjk1M7PWqZoQJL2xyqJtJBERv2pRTNZN+vqqtwBqluXLYenSNeePK3IBOwQf/CCsqvDU13Hj4Dvfac423GLKelit/3GvrbEsSD2XbaRrx0ms1KposOVNHll91arKSaZSkhiuUoupSvPNulzVhBAR72pnIGZm1llNvia3nlKkvNGMMst73wuVnrshwUknweOPpx8z6ygnhNGsSHmjGWWWiHTyrzS/9LvVpRwzq6tIPwQzMxsFhtPKCMCtjHpBO1q81CsHldR6VOvq1fDUU7W3U690VWRfi2ynUe1qMWXWAm5lNJK1o8VLvXJQs9QrXRXZ1zFjWl+aaleLKbMWcCsjMzMDCt5UlvSfwAuACaV5EfG54W5U0kHAccDzgZ0jYuFw1zViNaPc889/pp9qinQ6K1JmafbVQCXtKPe4U5mNckWeh3ASsA6wD3AKaRyjaxrc7o3AG4EfNLiekatZ5Z5a5ZwiJ7lmlVmaUVZqdWnKncpslCvSyuhlEXEI8FhEHA/sBmzTyEYj4paIuK2RdZiZWXMVKRn9O/9eIWlT4FFgk9aFNJCkw4HDAbbYYIN2bXbkaEc5p4hmxFFrHUXKX+PGVW+p1CztGPvJrEWK/E84V9Ik4KvAtaQWRqfU+5Cki4HnVVh0bET8pmiAEXEycDLArGnTuuTs1gb16v9Qv+Zdr9zTjJr5Bhs03qqmSFmq3naKxLvJJrVP1s3oMe17DdbDiiSEr0TEk8BZks4l3VheWe9DEbFfo8GNevVq5o3WvNtZM29H09R66u2ve0zbKFckIfwJ2BEgJ4YnJV1bmmct1OgJsxklknrrKFoiqbUvReJsRylGgmcqPB22UjIzG4Fq9VR+HrAZsLakl5AejgOwLqnV0bBJOhD4Dunpa7+TdF1EvKqRdY44zWjd04wx/uuto0iJZOzYxrZRdDuNWm+91ncqc9NW62K1viq+CpgDbA58o2z+E0BD/3Ij4mzg7EbWYT3EpZh+btpqXaxWT+UfAz+W9KaIOKuNMZmZWQcUKSZfJWkesGlEHCBpJrBbRMxrcWwjW73SQTvq6u1qItkttfl6++smozbKFUkIP8o/x+bpxcDPASeERtQrHbSjrt6umnU7avNF1Ntf1/BtlCvSU3nDiPgFsBogIlYBFb7umZlZLytyhfAvSZNJHdKQtCvg5x32gm5p0eJSTD8fC+tiRRLCR4FzgK0lXUVqKvrmlkZlzdEtLVpciunnY2FdrG5CiIhrJe0FbEvqi3BbRDzd8sjMzKytigx/PQF4H7AHqWx0haSTIqLu8BVWg0sHZtZlipSMTgOWkXoWA/wX8BPgoFYFNSq4dGBmXaZIQtg+ImaWTV8q6eZWBWRmZp1RJCFcK2nXiPgzgKRdAD/yshe4LGVmQ1AkIbwU+KOke/L0FsBtkm4AIiJe1LLorDEuS5nZEBRJCPu3PAozM+u4Is1O725HIGZm1llFhq4wM7NRwAnBzMwAJwQzM8ucEMzMDHBCMDOzzAnBzMwAJwQzM8ucEMzMDHBCMDOzzAnBzMwAJwQzM8ucEMzMDHBCMDOzzAnBzMwAJwQzM8ucEMzMDHBCMDOzzAnBzMwAJwQzM8ucEMzMDHBCMDOzzAnBzMyADiUESV+VdKukv0o6W9KkTsRhZmb9OnWFcBGwfUS8CFgMHN2hOMzMLOtIQoiICyNiVZ78M7B5J+IwM7N+3XAP4VDg99UWSjpc0kJJCx9evryNYZmZjS7jWrViSRcDz6uw6NiI+E1+z7HAKuCMauuJiJOBkwFmTZsWLQjVzMxoYUKIiP1qLZc0B3gNsG9E+ERvZtZhLUsItUjaH/gEsFdErOhEDGZmNlCn7iF8F+gDLpJ0naSTOhSHmZllHblCiIj/6MR2zcysum5oZWRmZl3ACcHMzAAnBDMzy5wQzMwMcEIwM7PMCcHMzAAnBDMzy9RLo0ZIehi4u9NxDNGGwCOdDqIFRuJ+jcR9Au9Xr2nFfk2LiCn13tRTCaEXSVoYEbM6HUezjcT9Gon7BN6vXtPJ/XLJyMzMACcEMzPLnBBa7+ROB9AiI3G/RuI+gfer13Rsv3wPwczMAF8hmJlZ5oRgZmaAE0JbSPqqpFsl/VXS2ZImdTqmRkk6SNJNklZL6vmmf5L2l3SbpNslfarT8TSDpB9KekjSjZ2OpZkkTZV0qaSb87/BozodUzNImiDpGknX5/06vt0xOCG0x0XA9hHxImAxcHSH42mGG4E3AvM7HUijJI0FvgccAMwE3iZpZmejaopTgf07HUQLrAI+FhEzgV2B94+Qv9eTwMsj4sXADsD+knZtZwBOCG0QERdGxKo8+Wdg807G0wwRcUtE3NbpOJpkZ+D2iLgjIp4CzgRe3+GYGhYR84F/djqOZouIByLi2vx6GXALsFlno2pcJMvz5Pj809ZWP04I7Xco8PtOB2EDbAYsKZu+lxFwghkNJE0HXgJc3dlImkPSWEnXAQ8BF0VEW/erI89UHokkXQw8r8KiYyPiN/k9x5Iud89oZ2zDVWSfzDpF0kTgLODDEfFEp+Nphoh4Btgh32c8W9L2EdG2e0BOCE0SEfvVWi5pDvAaYN/okc4f9fZpBLkPmFo2vXmeZ11K0nhSMjgjIn7V6XiaLSKWSrqUdA+obQnBJaM2kLQ/8AngdRGxotPx2BoWADMkbSnpOcDBwDkdjsmqkCRgHnBLRHyj0/E0i6QppRaIktYGXgHc2s4YnBDa47tAH3CRpOskndTpgBol6UBJ9wK7Ab+TdEGnYxqufMP/A8AFpBuUv4iImzobVeMk/Qz4E7CtpHslHdbpmJpkd+AdwMvz/6frJL2600E1wSbApZL+SvqSclFEnNvOADx0hZmZAb5CMDOzzAnBzMwAJwQzM8ucEMzMDHBCMDOzzAnBuo6kvSWt0dyu2vwmbO8N5YOjSbqs3giuOZbHJZ1X533HNCvOvL7l9d9V8/NzJH03v36vpEOaENNdkjaUtHZuAvqUpA0bXa+1nxOCGbyBNMrpUF0REfXavzc1IQyFkqr/xyPipIg4rVnbi4h/R8QOwP3NWqe1lxOCDZmk50r6XR63/UZJb83zXyrpckmLJF0gaZM8/zJJ38rfHm+UtHOev7OkP0n6i6Q/Stp2iDH8MI8f/xdJr8/z50j6laTzJf1N0lfKPnOYpMX5M/8n6buSXga8Dvhqjm/r/PaD8vsWS9qzQDybSJpfto97SvoSUPrWfEZ+36/z8blJ0uFln18u6Yv5mP5Z0sZ5/pb5GN0g6Qtl758o6RJJ1+Zlpf2frvRch9NIQx5MlfSu0n6TOnWV1nGcpI9L2rSsg9d1kp6RNC33nD1L0oL8s3v+3GRJF+Z9OAVQ0b+bdbmI8I9/hvQDvAn4v7Lp9UhD9f4RmJLnvRX4YX59Wen9wGzgxvx6XWBcfr0fcFZ+vTdwboXtPjsfOAH47/x6Euk5E88F5gB35JgmAHeTxinaFLgL2CDHegXw3fz5U4E3l23nMuDr+fWrgYtrxZKnP0Ya9A9gLNCXXy8f9LkN8u+1SSfsyXk6gNfm118BPp1fnwMckl+/v7Q+0jhk6+bXGwK3k07M04HVwK552SbAPcAU4DnAVWX7fRzw8UHxvZ/UUxvgp8Ae+fUWpKEiAL4NfCa//s8c+4Zl67irfNo/vfPjwe1sOG4Avi7py6ST4hWStge2Jw3PAemk+EDZZ34GaYx+SesqjdnSB/xY0gzSSWX8EGJ4JfA6SR/P0xNIJy2ASyLicQBJNwPTSCfNyyPin3n+L4Ftaqy/NGDaItJJtp4FwA+VBl37dURcV+V9H5J0YH49FZgBPAo8BZTujywijWMD6Rv9m/LrnwBfzq8FnCBpNikBbAZsnJfdHRF/zq93AS6LiIcBJP2cKvudrwDeA+yRZ+0HzMx/T4B1lUYYnU16OBIR8TtJj1XZV+sxTgg2ZBGxWNKOpG/PX5B0CXA2cFNE7FbtYxWmPw9cGhEHKo1rf9kQwhDwphj0kB5Ju5CePFXyDMP7d15aR6HP50Q3m/SN+VRJ34hB9XlJe5NOsrtFxApJl5ESGcDTkb9eV9hmpfFl3k761v/SiHha0l1l6/pXvXgHy+W9eaQBGEs3rseQrjRWDnrvUFdvPcL3EGzIJG0KrIiI04GvAjsCtwFTJO2W3zNe0gvKPla6z7AH8Hj+Br8e/cNMzxliGBcAH1Q+O0l6SZ33LwD2krS+pHH0f+sGWEa6Whk2SdOAByPi/4BTSMcE4Ol81QBpfx/LyWA70uMf67mKNPoqpCRQsh7wUE4G+5Cugiq5mrTfk3McB1WIfTzwS+CTEbG4bNGFwAfL3rdDfjkf+K887wBg/QL7YT3ACcGG44XANUpPdvos8IVIj558M/BlSdcD1wEvK/vMSkl/AU4CSqNufgWYm+cP9Vv850klpr9KuilPVxUR95HuO1xDOsneBTyeF58J/E++Ob115TXUtTdwfd6XtwLfyvNPzjGeAZwPjJN0C/Al0uNU6zmK9MzgGxj4FLczgFl5/iFUGSY5Ih4g3Sv4E2m/b6nwtpcBs4Djy24sbwp8KG/jr7n09t78/uOB2fm4v5F0j8JGAI92ai2XSyMfj4iFHY5jYkQsz1cIZ5Nuep89zHXtTdqn1zQzxpEgl69mRcQjnY7FhsZXCDaaHJevam4E7gR+3cC6ngK2V52OaaOJcsc00pXb6k7HY0PnKwQzMwN8hWBmZpkTgpmZAU4IZmaWOSGYmRnghGBmZtn/B67XCaTmUFUVAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f515085b8d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEKCAYAAAA4t9PUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3X2cnGV97/HPd2aTEBQIDxFhE02UFIuARALioaUKhcTWkpRCCaUVPRyptbS1Fo5QX8UjrQWLFeUcRCOC0FIRAWNsgwF50pctmA0JhACBGKJkCWZ5CFiJSXbzO3/c14TJZOaeex8mM8l+36/XvPae637Y39yb3V+uh/u6FBGYmZkNVandAZiZ2a7NicTMzIbFicTMzIbFicTMzIbFicTMzIbFicTMzIbFicTMzIbFicTMzIbFicTMzIalq90B7AwHHHBATJkypd1hmJntUpYsWfJ8RExsdtyoSCRTpkyhp6en3WGYme1SJP20yHFu2jIzs2FxIjEzs2FxIjEzs2FxIjEzs2FxIjEzs2EZFaO2hmL+0l6uWLSSZzds5OAJ47lw5qHMmd7d7rDMzDqOE0kd85f2cvHty9m4ZQCA3g0bufj25QBOJmZmNVratCVplqSVklZJuqjO/o9LekzSI5LulvTmqn3nSHoqvc6pKj9a0vJ0zaskaaTjvmLRym1JpGLjlgGuWLRypL+Vmdkur2WJRFIZuBp4H3AYcJakw2oOWwrMiIgjgVuBf0rn7gd8CngXcCzwKUn7pnOuAT4MTEuvWSMd+7MbNg6q3MxsNGtljeRYYFVErI6IzcDNwOzqAyLi3oh4Nb19AJiUtmcCd0XEixHxEnAXMEvSQcDeEfFARARwIzBnpAM/eML4QZWbmY1mrUwk3cAzVe/XprJGzgXuaHJud9puek1J50nqkdTT19c3qMAvnHko48eUtysbP6bMhTMPHdR1zMxGg44Y/ivpj4EZwBUjdc2ImBcRMyJixsSJTecc286c6d1cdtoRjB+T3Z7uCeO57LQj3NFuZlZHK0dt9QKTq95PSmXbkfTbwCeB34qITVXnvqfm3PtS+aSa8h2uORLmTO/m7ifW82jvy9x7wXuaHm9mNlq1skayGJgmaaqkscBcYEH1AZKmA18BTo2I9VW7FgGnSNo3dbKfAiyKiHXAK5KOS6O1PgB8p1UfoKsk+rdubdXlzcx2Cy2rkUREv6TzyZJCGbguIlZIuhToiYgFZE1Zrwe+lUbx/iwiTo2IFyX9PVkyArg0Il5M2x8Fvg6MJ+tTuYMWKZeE84iZWb6WPpAYEQuBhTVll1Rt/3bOudcB19Up7wEOH8EwGyrLNRIzs2Y6orO9U5XLYsB5xMwslxNJjq6SGHCNxMwslxNJjpJE/9ZodxhmZh3NiSRHV0lsdSIxM8vlRJKjXHKNxMysGSeSHOWSGHAiMTPL5USSo6skBsKJxMwsjxNJjlJJROB+EjOzHE4kObpK2ZpZrpWYmTXmRJKjXMpuj/tJzMwacyLJUU53xyO3zMwacyLJ4RqJmVlzTiQ5tvWROJGYmTXkRJKjlBKJZwA2M2vMiSRHpUbiPGJm1pgTSY6yXCMxM2vGiSRH2TUSM7OmnEhydJVdIzEza8aJJEdJHrVlZtZMSxOJpFmSVkpaJemiOvtPkPSQpH5Jp1eVv1fSsqrXryTNSfu+Lunpqn1HtSp+T5FiZtZcV6suLKkMXA2cDKwFFktaEBGPVR32M+CDwAXV50bEvcBR6Tr7AauAO6sOuTAibm1V7BWVPpL+AScSM7NGWpZIgGOBVRGxGkDSzcBsYFsiiYg1aV9eJ8TpwB0R8WrrQq2v7AcSzcyaamXTVjfwTNX7talssOYC36gp+4ykRyRdKWlcvZMknSepR1JPX1/fEL5tVSJx05aZWUMd3dku6SDgCGBRVfHFwNuAY4D9gE/UOzci5kXEjIiYMXHixCF9f9dIzMyaa2Ui6QUmV72flMoG4w+Bb0fElkpBRKyLzCbgerImtJZwH4mZWXOtTCSLgWmSpkoaS9ZEtWCQ1ziLmmatVEtBkoA5wKMjEGtdXWn2361u2jIza6hliSQi+oHzyZqlHgduiYgVki6VdCqApGMkrQXOAL4iaUXlfElTyGo099dc+iZJy4HlwAHAP7TqM3g9EjOz5lo5aouIWAgsrCm7pGp7MVmTV71z11Cncz4iThzZKBurrEfiNdvNzBrr6M72duvaNo28E4mZWSNOJDlemyLFc22ZmTXiRJKjMmnjgPOImVlDTiQ5Sl6PxMysKSeSHF6z3cysOSeSHH6y3cysOSeSHE4kZmbNOZHk8HokZmbNOZHkcI3EzKw5J5IcnrTRzKw5J5IclUTiSRvNzBpzIslRmf3XU6SYmTXmRJIj5RH3kZiZ5XAiyVGpkTiRmJk15kSSI3WRuGnLzCyHE0kOSZRL8uy/ZmY5nEiayBJJu6MwM+tcTROJpLKke3dGMJ2oLNdIzMzyNE0kETEAbJW0z2AvLmmWpJWSVkm6qM7+EyQ9JKlf0uk1+wYkLUuvBVXlUyU9mK75TUljBxvXYHS5RmJmlqvomu3/DSyXdBfwy0phRPxloxMklYGrgZOBtcBiSQsi4rGqw34GfBC4oM4lNkbEUXXKPwtcGRE3S/oycC5wTcHPMWjlsmskZmZ5iiaS29NrMI4FVkXEagBJNwOzgW2JJCLWpH2F/lJLEnAi8Eep6Abg/9DKRCJ51JaZWY5CiSQibkhNSL+WilZGxJYmp3UDz1S9Xwu8axCx7SGpB+gHLo+I+cD+wIaI6K+6Zvcgrjlo5ZI8RYqZWY5CiUTSe8j+978GEDBZ0jkR8YPWhcabI6JX0luAeyQtB14uerKk84DzAN70pjcNOYhySZ600cwsR9Hhv/8MnBIRvxURJwAzgSubnNMLTK56PymVFRIRvenrauA+YDrwAjBBUiUBNrxmRMyLiBkRMWPixIlFv+0OsuG/TiRmZo0UTSRjImJl5U1EPAmMaXLOYmBaGmU1FpgLLGhyDgCS9pU0Lm0fABwPPBYRAdwLVEZ4nQN8p+BnGJKukrywlZlZjqKJpEfStZLek15fBXryTkj9GOcDi4DHgVsiYoWkSyWdCiDpGElrgTOAr0hakU7/9fQ9HyZLHJdXjfb6BPBxSavI+ky+VvzjDl6p5M52M7M8RUdt/Rnw50BluO8PgS81OykiFgILa8ouqdpeTNY8VXvefwJHNLjmarIRYTtFV0lsdSIxM2uoaSJJz4NcFxFnA59vfUidpVwquUZiZpaj6JPtb271E+SdqlzyNPJmZnmKNm2tBn6UpiqpfrJ9t6+hlEslJxIzsxxFE8lP0qsE7NW6cDpPWa6RmJnlKdpHsldE1JsPa7fXVSrR77m2zMwaKtpHcvxOiKUjlUvCecTMrLGiTVuVqdy/xfZ9JIOdyHGXUy6JTf0D7Q7DzKxjFU0ke5BNT3JiVVkw+BmBdznlkvBUW2ZmjRWd/fdDrQ6kU3V5zXYzs1y5fSSSbqna/mzNvjtbFVQnKXn2XzOzXM0626dVbZ9cs2/oU+ruQrq8HomZWa5miSTvL+io+Ota9qSNZma5mvWR7ClpOlnCGZ+2lV7jWx1cJ/B6JGZm+ZolknW8NlHjc2w/aeNzLYmowziRmJnly00kEfHenRVIpyrLicTMLE/Rha22kTSvFYF0qq6y+0jMzPIMOpEAM0Y8ig5W9sJWZma5hpJI1o94FB2sLNdIzMzyDDqRRMSsVgTSqcqlkmskZmY5cjvbJX2XnOdFIuLUJufPAr4IlIFrI+Lymv0nAF8AjgTmRsStqfwo4Bpgb2AA+ExEfDPt+zrwW8DL6TIfjIhleXEMh/tIzMzyNRv++7n09TTgjcC/pvdnAT/POzGtY3I12RPxa4HFkhZExGNVh/0M+CBQu9bJq8AHIuIpSQcDSyQtiogNaf+FlaTTaiWP2jIzy9Vs+O/9AJL+OSKqO9m/K6mnybWPBVZFxOp0jZuB2cC2RBIRa9K+7WZFjIgnq7aflbSebEqWDexkXSUx4ClSzMwaKtpH8jpJb6m8kTQVeF2Tc7qBZ6rer01lgyLpWGAs2VK/FZ+R9IikKyWNG+w1B6OUHkgMJxMzs7qKJpK/Bu6TdJ+k+4F7gY+1LqyMpIOAfwE+FBGVWsvFwNuAY4D9gE80OPc8ST2Sevr6+oYcQ1dJgNdtNzNrpOh6JN+TNI3sDzjAExGxqclpvcDkqveTUlkhkvYG/gP4ZEQ8UBXLurS5SdL17Ni/UjluHjAPYMaMGUPOAuVKIokovAqYmdloUqhGImlP4ELg/Ih4GHiTpPc3OW0xME3SVEljgbnAgoLfbyzwbeDG2k71VEtBkoA5wKNFrjlUZddIzMxyFW3auh7YDLw7ve8F/iHvhIjoB84HFgGPA7dExApJl0o6FUDSMZLWAmcAX5G0Ip3+h8AJwAclLUuvo9K+myQtB5YDBzSLY7jctGVmlq9oa81bI+JMSWcBRMSrqUaQKyIWAgtryi6p2l5M1uRVe96/8tpQ49p9J9YrbxXXSMzM8hWtkWyWNJ70cKKktwLN+kh2C5VE4ocSzczqK1oj+RTwPWCypJuA48keJNztVRKJp0kxM6uvaSJJTVhPkD3dfhzZ6oh/FRHPtzi2jlCWayRmZnmaJpKICEkLI+IIsuG4o4r7SMzM8hXtI3lI0jEtjaRDdZWdSMzM8hTtI3kXcLaknwK/JGveiog4smWRdYiSm7bMzHIVTSQzWxpFB+sqZZU210jMzOorOkXKTwEkvQHYo6URdRj3kZiZ5Ss6Rcqpkp4CngbuB9YAd7Qwro7hRGJmlq9oZ/vfkw39fTIipgInAQ/kn7J76KqatNHMzHZUNJFsiYgXgJKkUkTcC8xodtLu4LUaydYmR5qZjU5FO9s3SHo98AOySRPXk43e2u1tmyJlwDUSM7N6itZIZgMbyRa4+h7ZaoW/16qgOknZTVtmZrmKjtqqrn3c0KJYOpI7283M8hVKJJJ+QZr5l2z99DHALyNi71YF1ik8+6+ZWb6iNZK9KttpEsfZZKO4dntdnv3XzCxX0T6SbSIzn1HytLunSDEzy1e0aeu0qrclsqG/v2pJRB2mMmmjayRmZvUVHf5bPUKrn+zJ9tkjHk0H6nIfiZlZrkJNWxHxoarXhyPiMxGxvtl5kmZJWilplaSL6uw/QdJDkvolnV6z7xxJT6XXOVXlR0tanq55VZG144ej0rTlUVtmZvUVbdq6Km9/RPxlnXPKwNXAycBaYLGkBRHxWNVhPyNbsveCmnP3I1vedwbZaLEl6dyXgGuADwMPAguBWbRw3i/P/mtmlq9oZ/sewDuBp9LrKLJhwEvSq55jgVURsToiNgM3U9McFhFrIuIRoHb+kZnAXRHxYkoedwGzJB0E7B0RD0READcCcwp+hiFJecSJxMysgaJ9JEcCvxER/QCSvgz8MCI+knNON/BM1fu1ZAtkFVHv3O70WlunvGUqNRL3kZiZ1Ve0RrIvUP3w4etTWceSdJ6kHkk9fX19Q76Op0gxM8tXNJFcDiyV9HVJNwAPAf/Y5JxeYHLV+0mprIhG5/am7abXjIh5ETEjImZMnDix4Lfd0bZEMuDZf83M6ik6aut6smapbwO3Ae+OiGZzbi0GpkmaKmksMBdYUDCuRcApkvaVtC9wCrAoItYBr0g6Lo3W+gDwnYLXHJLXaiSt/C5mZruu3EQi6c2S9gGIiOeAV8gWtfqjlBwaSv0p55MlhceBWyJihaRLJZ2arn+MpLXAGcBXJK1I575ItpjW4vS6NJUBfBS4FlhFNgtxS1dq7PJ6JGZmuZp1tt8C/D7wsqSjgG8BlwHvAL4E/K+8kyNiIdkQ3eqyS6q2F7N9U1X1cdcB19Up7wEObxL3iPGkjWZm+ZolkvER8Wza/mPguoj4Z0klYFlrQ+sMZU/aaGaWq1kfSfVT4ycCdwNExKhp5yl70kYzs1zNaiT3SLoFWEc23PcegPRg4OYWx9YRSiUh+YFEM7NGmiWSjwFnAgeRPZC4JZW/EfhkKwPrJF0lOZGYmTWQm0jSNCQ3V5dJen9E/HtLo+owJTmRmJk1MuiFrYBLRzyKDtdVkvtIzMwaGEoiaem07Z2o7KYtM7OGhpJI/nTEo+hwTiRmZo0VXY+kDPwuMAXokvQbABHx+daF1jnKpZInbTQza6DoNPLfJVujfTk7rh2y2yuXYMCTbZmZ1VU0kUyKiCNbGkkH6yqV3NluZtZA0T6SOySd0tJIOli5JLa6acvMrK6iNZIHgG+nOba2kI3ciojYO/+03UPZw3/NzBoqmkg+D7wbWJ4eUhxVslFbo65ryMyskKJNW88Aj47GJAKeIsXMLE/RGslq4D5JdwCbKoWjZfivp0gxM2usaCJ5Or3Gpteo0lV2IjEza6RQIomIT7c6kE7mznYzs8aKPtl+L7DDX9KIOHHEI+pAZTdtmZk1VLRp64Kq7T2APwD6m50kaRbwRaAMXBsRl9fsHwfcCBwNvACcGRFrJJ0NXFh16JHAOyNimaT7yNZH2Zj2nRIR6wt+jiHxXFtmZo0VbdpaUlP0I0k/zjsnzc91NXAysBZYLGlBRDxWddi5wEsRcYikucBnyZLJTcBN6TpHAPMjonqN+LMjoqdI7COhXBKb+z3818ysnkLDfyXtV/U6INU09mly2rHAqohYHRGbyRbIml1zzGzghrR9K3CSpNpp6s+iZnGtnc19JGZmjRVt2lrCa30k/cAastpEnm6y508q1gLvanRMRPRLehnYH3i+6pgz2TEBXS9pALgN+IdWPt8yf2kvP376RTb1b+X4y+/hwpmHMmd6d6u+nZnZLie3RiLpGElvjIipEfEW4NPAE+n1WN65I0HSu4BXI+LRquKzI+II4DfT608anHuepB5JPX19fUP6/vOX9nLx7cvZlJq1ejds5OLblzN/ae+Qrmdmtjtq1rT1FWAzgKQTgMvImqJeBuY1ObcXmFz1flIqq3uMpC6y5rIXqvbPBb5RfUJE9KavvwD+jawJbQcRMS8iZkTEjIkTJzYJtb4rFq1k45aB7co2bhngikUrh3Q9M7PdUbNEUo6IF9P2mcC8iLgtIv4OOKTJuYuBaZKmShpLlhQW1ByzADgnbZ8O3FNppkoTRP4hVf0jkrokHZC2xwDvBx6lRZ7dsHFQ5WZmo1HTRJJqCgAnAfdU7cvtX4mIfuB8YBHwOHBLRKyQdKmkU9NhXwP2l7QK+DhwUdUlTgCeiYjVVWXjgEWSHgGWkdVovtrkMwzZwRPGD6rczGw0atbZ/g3gfknPkz238UMASYeQNW/lioiFwMKaskuqtn8FnNHg3PuA42rKfkn2zMlOceHMQ7n49uXbNW+NH1PmwpmH7qwQzMw6XrNaxWck3U32AOCdVaOjSsBftDq4dquMzvr0d1fw0qtbeMNe4/jb3/l1j9oyM6vSdPhvRDxQp+zJ1oTTeeZM7+bAvffgrK8+wJVnHsXxhxzQ7pDMzDpK0fVIRrUD9x4HwM9f+VWbIzEz6zxOJAUcuPceAPz8lU1NjjQzG32cSAp43bgu9hrX5RqJmVkdTiQFHbjPHk4kZmZ1OJEUdODe43jOicTMbAdOJAUduNcerHcfiZnZDpxICjpwnz1Y/4tfsdXTyZuZbceJpKAD9xrHloHgxVc3tzsUM7OO4kRS0Bv3qQwBdj+JmVk1J5KC3pCeJXE/iZnZ9pxICqo8lOiRW2Zm23MiKegNe3maFDOzepxICvqPR9ZREnzh+09x/OX3eLldM7PEiaSAytrtlZG/XrvdzOw1TiQFeO12M7PGnEgK8NrtZmaNOZEU4LXbzcwaa2kikTRL0kpJqyRdVGf/OEnfTPsflDQllU+RtFHSsvT6ctU5R0tans65SpJa+RkgW7t9/JjydmVeu93MLNOyRCKpDFwNvA84DDhL0mE1h50LvBQRhwBXAp+t2veTiDgqvT5SVX4N8GFgWnrNatVnqJgzvZvLTjuCgydkz5K8blyZy047wmu3m5nR2hrJscCqiFgdEZuBm4HZNcfMBm5I27cCJ+XVMCQdBOwdEQ9ERAA3AnNGPvQdzZnezX9edBJHv3lfDjtobycRM7OklYmkG3im6v3aVFb3mIjoB14G9k/7pkpaKul+Sb9ZdfzaJtcEQNJ5knok9fT19Q3vk1Q5onsfHu19hQHPAmxmBnRuZ/s64E0RMR34OPBvkvYezAUiYl5EzIiIGRMnThyxwI6ctA8btwywav1/j9g1zcx2Za1MJL3A5Kr3k1JZ3WMkdQH7AC9ExKaIeAEgIpYAPwF+LR0/qck1W6rvF9kUKTO/8AM/4W5mRmsTyWJgmqSpksYCc4EFNccsAM5J26cD90RESJqYOuuR9BayTvXVEbEOeEXScakv5QPAd1r4GbYzf2kvX/j+U9ve+wl3M7MWJpLU53E+sAh4HLglIlZIulTSqemwrwH7S1pF1oRVGSJ8AvCIpGVknfAfiYgX076PAtcCq8hqKne06jPUyp5w37pdmZ9wN7PRrquVF4+IhcDCmrJLqrZ/BZxR57zbgNsaXLMHOHxkIy3GT7ibme2oUzvbO5KfcDcz25ETySD4CXczsx21tGlrd1N5CPGKRSvp3bCRckn84+8f7ocTzWxUc41kkOZM7+ZHF53IpbPfzsDW4Kg37dvukMzM2sqJZIi29Gejt977ufv8PImZjWpOJEMwf2kvn7vzyW3v/TyJmY1mTiRD4BUTzcxe40QyBH6exMzsNU4kQ9DouZGS5OYtMxt1nEiGoN7zJAADEe4rMbNRx4lkCCorJpbrrMHlvhIzG22cSIZozvRutkb9xa3cV2Jmo4kTyTC4r8TMzIlkWNxXYmbmRDIs7isxM3MiGba8vpLeDRtdKzGz3Z4TyQjIW4/kY99cxvRL73RCMbPdlhPJCGjUV1Lx0qtbnFDMbLelaNAsszuZMWNG9PT0tPR7zF/ay8e+uazQsXuOKTFuTJkNr27h4AnjuXDmoV7TxMw6jqQlETGj6XGtTCSSZgFfBMrAtRFxec3+ccCNwNHAC8CZEbFG0snA5cBYYDNwYUTck865DzgIqDyscUpErM+LY2ckEoDjL7+H3iE+Q1ISbA0oSwxEDPpr94TxvPdtE7n3ib5s0a06+2sT1vylvVyxaCXPbtjIwQ3Or3detaLXyIut3veqXLdRLLXfN++82q8Txo9BymqKg4ml9jM2ej+U+5z386y+LlAolkb3v8j9yju+9jMV+TkX/SyD+RkO59/QcO5/5R4M59oj9bs92Ps9lP+stj2RSCoDTwInA2uBxcBZEfFY1TEfBY6MiI9Imgv8fkScKWk68POIeFbS4cCiiOhO59wHXBARhTPDzkok85f2cvHty3eYGbiTVBKWgMH85GsT3YTxY9jcP8CrW7a2KtRd2lDvcyfbHT/TYO05JusN2NX+3Y8fU+ay044YdDIpmkha2UdyLLAqIlZHxGbgZmB2zTGzgRvS9q3ASZIUEUsj4tlUvgIYn2ovHa0yHHjC+DHtDqWhrekvwGD/EFTOG0j/8diwccsu98u0Mw31Pney3fEzDdarW7bukv/uW/04QisTSTfwTNX7tams7jER0Q+8DOxfc8wfAA9FxKaqsuslLZP0d1KdhzgASedJ6pHU09fXN5zPMShzpnez7FOn8IUzj+rohGJmo0srp27q6FFbkt4OfBb406risyPiCOA30+tP6p0bEfMiYkZEzJg4cWLrg63hhGJmnSTvMYXhamUi6QUmV72flMrqHiOpC9iHrNMdSZOAbwMfiIifVE6IiN709RfAv5E1oXWs6oTSPWE8AiaMH7OtrdXMrNXGjylvG+DQCl0tu3LWuT5N0lSyhDEX+KOaYxYA5wD/BZwO3BMRIWkC8B/ARRHxo8rBKdlMiIjnJY0B3g98v4WfYcTMmd69Q0fXcEanVL426vjs5I7R2o77VsXYaCRc3qitnX2/dta92FUVHc041PvWyvs/3JGYIxXLvnuO4VO/9/aWPmLQskQSEf2SzgcWkQ3/vS4iVki6FOiJiAXA14B/kbQKeJEs2QCcDxwCXCLpklR2CvBLYFFKImWyJPLVVn2GVquXXIai0VDY2v2NhgUPZmho7R/jyrMwza7RbBhi0eG+RYc/DufZnKJDYosMvR3KkMxmw42LDANuNnS56LDqokPKmw3HLhL7cH+GQx0yPpj7v0/Nv/vBXnuwBnNfWxlHM34g0czM6uqE4b9mZjYKOJGYmdmwOJGYmdmwOJGYmdmwOJGYmdmwjIpRW5L6gJ8O8rQDgOdbEM5Icowjo9Nj7PT4wDGOlE6L8c0R0XRqkFGRSIZCUk+RYW/t5BhHRqfH2OnxgWMcKbtCjPW4acvMzIbFicTMzIbFiaSxee0OoADHODI6PcZOjw8c40jZFWLcgftIzMxsWFwjMTOzYXEiqUPSLEkrJa2SdFEHxDNZ0r2SHpO0QtJfpfL9JN0l6an0dd8OiLUsaamkf0/vp0p6MN3Lb0oa2+b4Jki6VdITkh6X9O5Ou4+S/jr9nB+V9A1Je7T7Pkq6TtJ6SY9WldW9b8pclWJ9RNI72xjjFeln/Yikb6clKir7Lk4xrpQ0s10xVu37G0kh6YD0vi33cSicSGpIKgNXA+8DDgPOknRYe6OiH/ibiDgMOA748xTTRcDdETENuDu9b7e/Ah6vev9Z4MqIOAR4CTi3LVG95ovA9yLibcA7yGLtmPsoqRv4S2BGRBxOtlzCXNp/H78OzKopa3Tf3gdMS6/zgGvaGONdwOERcSTwJHAxQPr9mQu8PZ3zpfS7344YkTSZbKmMn1UVt+s+DpoTyY6OBVZFxOqI2AzcDMxuZ0ARsS4iHkrbvyD749ed4rohHXYDMKc9EWbSqpa/C1yb3gs4Ebg1HdLWGCXtA5xAtg4OEbE5IjbQYfeRbJ2g8Wkhtz2BdbT5PkbED8jWDKrW6L7NBm6MzAPABEkHtSPGiLgzIvrT2wfIVmqtxHhzRGyKiKeBVeyE1VYb3EeAK4H/zfZrWbXlPg6FE8mOuoFnqt6vTWUdQdIUYDrwIHBgRKxLu54DDmxTWBVfIPtl2Jre7w9sqPpFbve9nAr0Aden5rdrJb2ODrqPaSnpz5H9z3QZI8UsAAAEWElEQVQd8DKwhM66jxWN7lun/g79T+COtN0xMUqaDfRGxMM1uzomxmacSHYhkl4P3AZ8LCJeqd4X2fC7tg3Bk/R+YH1ELGlXDAV0Ae8EromI6WQrbm7XjNUB93Ffsv+JTgUOBl5HnaaQTtPu+9aMpE+SNRHf1O5YqknaE/hb4JJmx3YyJ5Id9QKTq95PSmVtlZYXvg24KSJuT8U/r1R109f17YoPOB44VdIasubAE8n6IyakJhpo/71cC6yNiAfT+1vJEksn3cffBp6OiL6I2ALcTnZvO+k+VjS6bx31OyTpg8D7gbPjtecdOiXGt5L9p+Hh9LszCXhI0hvpnBibciLZ0WJgWholM5asQ25BOwNKfQ1fAx6PiM9X7VoAnJO2zwG+s7Njq4iIiyNiUkRMIbtn90TE2cC9wOnpsHbH+BzwjKRDU9FJwGN00H0ka9I6TtKe6edeibFj7mOVRvdtAfCBNOroOODlqiawnUrSLLLm1lMj4tWqXQuAuZLGSZpK1qH9450dX0Qsj4g3RMSU9LuzFnhn+rfaMfexqYjwq+YF/A7ZCI+fAJ/sgHh+g6zZ4BFgWXr9DlkfxN3AU8D3gf3aHWuK9z3Av6ftt5D9gq4CvgWMa3NsRwE96V7OB/bttPsIfBp4AngU+BdgXLvvI/ANsj6bLWR/7M5tdN8AkY18/AmwnGwEWrtiXEXWz1D5vfly1fGfTDGuBN7Xrhhr9q8BDmjnfRzKy0+2m5nZsLhpy8zMhsWJxMzMhsWJxMzMhsWJxMzMhsWJxMzMhsWJxGwQJA1IWlb1GrEJHiVNqTcrrFmn62p+iJlV2RgRR7U7CLNO4hqJ2QiQtEbSP0laLunHkg5J5VMk3ZPWk7hb0ptS+YFpfYyH0+t/pEuVJX1V2Xokd0oan45/q6TvSVoi6YeS3pbKz1C2bsnDkn7Qlg9vo54TidngjK9p2jqzat/LEXEE8P/IZkIG+L/ADZGth3ETcFUqvwq4PyLeQTbf14pUPg24OiLeDmwA/iCVzwP+IiKOBi4AvpTKLwFmpuucOtIf1qwIP9luNgiS/jsiXl+nfA1wYkSsThNsPhcR+0t6HjgoIrak8nURcYCkPmBSRGyqusYU4K7IFopC0ieAMWRJqY9sKo+KcRHx65K+TDbx3y3A7RHxQgs+tlku95GYjZxosD0Ym6q2B4DxZC0HG+r1zUTERyS9i2xBsSWSjnYysZ3NTVtmI+fMqq//lbb/k2w2ZICzgR+m7buBP4Nt69zv0+iika0987SkM9LxkvSOtP3WiHgwIi4hq7VMbnQds1ZxIjEbnNo+ksur9u0r6RGydev/OpX9BfChVP4naR/p63slLSdbAfGwJt/3bOBcSQ+T9adUln++InXwP0qWtGpX2TNrOfeRmI2A1EcyIyKeb3csZjubayRmZjYsrpGYmdmwuEZiZmbD4kRiZmbD4kRiZmbD4kRiZmbD4kRiZmbD4kRiZmbD8v8BUp/FDGCwsl4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f515015b450>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_decision_region(X_std, y, classifier=ada)\n",
    "\n",
    "plt.title(\"Adaline - Stochastic Gradient Descent\")\n",
    "plt.xlabel(\"sepal length [standardized]\")\n",
    "plt.ylabel(\"petal length [standardized]\")\n",
    "plt.legend(loc=\"upper left\")\n",
    "plt.show()\n",
    "\n",
    "plt.plot(range(1, len(ada.cost_) + 1), ada.cost_, marker='o')\n",
    "plt.xlabel(\"Epoches\")\n",
    "plt.ylabel(\"Sum-Squared-Error\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以发现，平均损失\\(average cost\\)下降的非常快，在第15次迭代后决策界和使用梯度下降的Adaline决策界非常相似。如果我们要在在线环境下更新模型参数，通过调用partial\\_fit方法即可，此时参数是一个训练样本，比如ada.partial\\_fit\\(X\\_std\\[0, :\\], y\\[0\\]\\)。\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
